[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:03:48 dist-test-slave-j4lq krb5kdc[2399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:03:48 dist-test-slave-j4lq krb5kdc[2399](info): set up 2 sockets
Sep 26 13:03:48 dist-test-slave-j4lq krb5kdc[2399](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 13:03:49.561475 2392 mini_kdc.cc:121] Time spent starting KDC: real 3.036s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260926 13:03:49.699535 2392 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.138s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260926 13:03:49.936092 2392 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.236s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260926 13:03:50.247255 2392 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.311s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/test-user.keytab.
W20260926 13:03:50.776019 2392 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.529s user 0.000s sys 0.001s
Sep 26 13:03:50 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:03:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:03:50.819515 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40575
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:50.901543 2412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:50.901744 2412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:50.901767 2412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:50.902823 2412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:03:50.902855 2412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:50.902870 2412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:50.902882 2412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:03:50.902894 2412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:03:50.904268 2412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:50.904459 2412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:50.904652 2412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:50.909888 2417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:50.910990 2420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:50.911021 2412 server_base.cc:1034] running on GCE node
W20260926 13:03:50.911170 2418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:50.911437 2412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:50.911682 2412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:50.912854 2412 hybrid_clock.cc:648] HybridClock initialized: now 1790427830912835 us; error 34 us; skew 500 ppm
Sep 26 13:03:50 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427830, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:50.915388 2412 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:03:50.915820 2412 webserver.cc:460] Webserver started at http://127.2.86.62:32927/ using document root <none> and password file <none>
I20260926 13:03:50.916005 2412 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:50.916045 2412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:50.916131 2412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:50.917236 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "125afdb8f9744552920935885fae37c7"
format_stamp: "Formatted at 2026-09-26 13:03:50 on dist-test-slave-j4lq"
server_key: "0156e3d535bf9c039f466a1f4de2dd2b"
server_key_iv: "0365fe4a3e11fdf376da43543e7a1833"
server_key_version: "encryptionkey@0"
I20260926 13:03:50.917538 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "125afdb8f9744552920935885fae37c7"
format_stamp: "Formatted at 2026-09-26 13:03:50 on dist-test-slave-j4lq"
server_key: "0156e3d535bf9c039f466a1f4de2dd2b"
server_key_iv: "0365fe4a3e11fdf376da43543e7a1833"
server_key_version: "encryptionkey@0"
I20260926 13:03:50.918803 2412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:50.919440 2427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:50.919602 2412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:50.919696 2412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "125afdb8f9744552920935885fae37c7"
format_stamp: "Formatted at 2026-09-26 13:03:50 on dist-test-slave-j4lq"
server_key: "0156e3d535bf9c039f466a1f4de2dd2b"
server_key_iv: "0365fe4a3e11fdf376da43543e7a1833"
server_key_version: "encryptionkey@0"
I20260926 13:03:50.919775 2412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:50.942273 2412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:50.944468 2412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:50.944645 2412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:50.948419 2412 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40575
I20260926 13:03:50.948458 2479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40575 every 8 connection(s)
I20260926 13:03:50.948850 2412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:03:50.949775 2480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:50.951985 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7: Bootstrap starting.
I20260926 13:03:50.952550 2480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:50.952816 2480 log.cc:824] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:50.953542 2480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7: No bootstrap required, opened a new log
I20260926 13:03:50.954140 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 2412
I20260926 13:03:50.954248 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:03:50.954877 2392 external_mini_cluster.cc:1428] Setting key 2b7cc9ff1f95b629b56c403567c8f701
I20260926 13:03:50.955205 2480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } }
I20260926 13:03:50.955372 2480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:50.955418 2480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125afdb8f9744552920935885fae37c7, State: Initialized, Role: FOLLOWER
I20260926 13:03:50.955607 2480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } }
I20260926 13:03:50.955713 2480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:03:50.955747 2480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:03:50.955787 2480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:50.956699 2480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } }
I20260926 13:03:50.956835 2480 leader_election.cc:302] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 125afdb8f9744552920935885fae37c7; no voters:
I20260926 13:03:50.957036 2480 leader_election.cc:288] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:03:50.957059 2485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:50.957165 2485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [term 1 LEADER]: Becoming Leader. State: Replica: 125afdb8f9744552920935885fae37c7, State: Running, Role: LEADER
I20260926 13:03:50.957247 2485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } }
I20260926 13:03:50.957309 2480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:03:50.957587 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "125afdb8f9744552920935885fae37c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } } }
I20260926 13:03:50.957732 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:50.957930 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 125afdb8f9744552920935885fae37c7. Latest consensus state: current_term: 1 leader_uuid: "125afdb8f9744552920935885fae37c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125afdb8f9744552920935885fae37c7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40575 } } }
I20260926 13:03:50.958035 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:50.958760 2492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:03:50.959460 2492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:03:50.961349 2492 catalog_manager.cc:1347] Generated new cluster ID: 8f772b081e2f4c3186e8c4b51d3c88c1
I20260926 13:03:50.961411 2492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:03:50 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:50.973107 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:50.955774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.2.86.62:40575)
0926 13:03:50.955974 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:50.955984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:50.956267 (+ 283us) server_negotiation.cc:407] Connection header received
0926 13:03:50.958425 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:50.958434 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:50.958498 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:50.958666 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:50.960721 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:50.962114 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:50.963655 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:50.964265 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:50.964361 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:50.968957 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:50.968962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:50.968962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:50.968973 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:50.971094 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:50.971553 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:50.971556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:50.971556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:50.971677 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:50.971967 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:50.971968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:50.971969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:50.972271 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:50.972429 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:50.972697 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 13:03:50.972832 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260926 13:03:50.974612 2492 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:03:50.975040 2492 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:03:50.980067 2492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 125afdb8f9744552920935885fae37c7: Generated new TSK 0
I20260926 13:03:50.980240 2492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:03:51.005859 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:51.086066 2506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:51.086256 2506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:51.086279 2506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:51.087320 2506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:51.087350 2506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:51.087375 2506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:03:51.088683 2506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:51.088850 2506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:51.089118 2506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:51.093724 2512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:51.093735 2511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:51.093731 2514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:51.094079 2506 server_base.cc:1034] running on GCE node
I20260926 13:03:51.094260 2506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:51.094485 2506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:51.095638 2506 hybrid_clock.cc:648] HybridClock initialized: now 1790427831095628 us; error 40 us; skew 500 ppm
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:51.098126 2506 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:03:51.098608 2506 webserver.cc:460] Webserver started at http://127.2.86.1:35711/ using document root <none> and password file <none>
I20260926 13:03:51.098781 2506 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:51.098820 2506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:51.098924 2506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:51.099952 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "40cbec39a6764fd09f04245042efef91"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "a3c8ecb70807ae11165750547dcd5ced"
server_key_iv: "191260a9b16c2242d6dadc49bf127fe3"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.100263 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "40cbec39a6764fd09f04245042efef91"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "a3c8ecb70807ae11165750547dcd5ced"
server_key_iv: "191260a9b16c2242d6dadc49bf127fe3"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.101569 2506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:03:51.102221 2521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.102417 2506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.102494 2506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "40cbec39a6764fd09f04245042efef91"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "a3c8ecb70807ae11165750547dcd5ced"
server_key_iv: "191260a9b16c2242d6dadc49bf127fe3"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.102545 2506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:51.110852 2506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:51.112627 2506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:51.112737 2506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:51.112986 2506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:51.113243 2506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:51.113281 2506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.113302 2506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:51.113314 2506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.118703 2506 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41621
I20260926 13:03:51.118777 2634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41621 every 8 connection(s)
I20260926 13:03:51.119067 2506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:03:51.120193 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 2506
I20260926 13:03:51.120258 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:03:51.120427 2392 external_mini_cluster.cc:1428] Setting key 89e2c69d222d843b3c7d7a7e57e776c7
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:51.129415 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.119770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:37563 (local address 127.2.86.62:40575)
0926 13:03:51.119852 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:51.119854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:51.120005 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:03:51.120578 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:51.120580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:51.120592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:51.120646 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:51.121800 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.122546 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.123826 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.124022 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.124101 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.126591 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:51.126592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:51.126592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:51.126598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:51.127941 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.128421 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.128421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.128422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.128468 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.128739 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.128739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.128739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.128843 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:51.128943 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:51.129276 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 13:03:51.129351 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
I20260926 13:03:51.129642 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.119726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40575 (local address 127.2.86.1:37563)
0926 13:03:51.119893 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:51.119925 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:51.120483 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:51.120711 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:51.120715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:51.121510 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:51.121703 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.121708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.122628 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.122629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.123759 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.123761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.124110 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.124111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.124238 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.124730 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:51.124738 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:51.126501 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:51.128022 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.128028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.128031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.128336 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.128533 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.128533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.128534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.128666 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.128959 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:51.128966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:51.129134 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 13:03:51.129261 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 13:03:51.129420 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260926 13:03:51.130231 2635 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40575
I20260926 13:03:51.130308 2635 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:51.130504 2635 heartbeater.cc:511] Master 127.2.86.62:40575 requested a full tablet report, sending...
I20260926 13:03:51.131032 2444 ts_manager.cc:194] Registered new tserver with Master: 40cbec39a6764fd09f04245042efef91 (127.2.86.1:41621)
I20260926 13:03:51.131783 2444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:37563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:03:51.147563 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:51.227862 2640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:51.228056 2640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:51.228080 2640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:51.229151 2640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:51.229184 2640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:51.229210 2640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:03:51.230562 2640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:51.230756 2640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:51.230926 2640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:51.235882 2646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:51.235893 2645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:51.235926 2648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:51.235944 2640 server_base.cc:1034] running on GCE node
I20260926 13:03:51.236230 2640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:51.236409 2640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:51.237566 2640 hybrid_clock.cc:648] HybridClock initialized: now 1790427831237553 us; error 25 us; skew 500 ppm
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:51.239992 2640 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:03:51.240484 2640 webserver.cc:460] Webserver started at http://127.2.86.2:36987/ using document root <none> and password file <none>
I20260926 13:03:51.240670 2640 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:51.240710 2640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:51.240794 2640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:51.241819 2640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "c49b285cfa064af588e4748e096294f5"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "d6817092e68719ffe05fd2cfff5217ed"
server_key_iv: "381ff5074a4df91cf19e4e8fcfadb1b5"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.242127 2640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "c49b285cfa064af588e4748e096294f5"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "d6817092e68719ffe05fd2cfff5217ed"
server_key_iv: "381ff5074a4df91cf19e4e8fcfadb1b5"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.243352 2640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:03:51.244002 2655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.244163 2640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:03:51.244247 2640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "c49b285cfa064af588e4748e096294f5"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "d6817092e68719ffe05fd2cfff5217ed"
server_key_iv: "381ff5074a4df91cf19e4e8fcfadb1b5"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.244319 2640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:51.256569 2640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:51.258685 2640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:51.258844 2640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:51.259092 2640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:51.259419 2640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:51.259479 2640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.259521 2640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:51.259549 2640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.265748 2640 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:44349
I20260926 13:03:51.266122 2640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:03:51.266213 2768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:44349 every 8 connection(s)
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:51.272318 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 2640
I20260926 13:03:51.272426 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:03:51.272596 2392 external_mini_cluster.cc:1428] Setting key fcab5ab8ccad33d5ca75f8e5d5783dc7
I20260926 13:03:51.275354 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.266881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:55479 (local address 127.2.86.62:40575)
0926 13:03:51.266951 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:51.266953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:51.267154 (+ 201us) server_negotiation.cc:407] Connection header received
0926 13:03:51.267652 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:51.267654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:51.267665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:51.267720 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:51.268627 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.269214 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.270227 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.270438 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.270514 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.272427 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:51.272428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:51.272429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:51.272437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:51.273988 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.274398 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.274399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.274400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.274460 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.274715 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.274716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.274716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.274826 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:51.274916 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:51.275181 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 13:03:51.275266 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:03:51.275553 2771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.266857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40575 (local address 127.2.86.2:55479)
0926 13:03:51.267037 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:51.267069 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:51.267547 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:51.267796 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:51.267800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:51.268345 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:51.268527 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.268532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.269316 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.269317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.270123 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.270125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.270518 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.270519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.270641 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.271075 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:51.271083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:51.272324 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:51.274073 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.274079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.274081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.274298 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.274525 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.274526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.274526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.274638 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.274909 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:51.274914 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:51.275053 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 13:03:51.275162 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 13:03:51.275310 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260926 13:03:51.276441 2769 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40575
I20260926 13:03:51.276540 2769 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:51.276703 2769 heartbeater.cc:511] Master 127.2.86.62:40575 requested a full tablet report, sending...
I20260926 13:03:51.277115 2444 ts_manager.cc:194] Registered new tserver with Master: c49b285cfa064af588e4748e096294f5 (127.2.86.2:44349)
I20260926 13:03:51.277493 2444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:55479
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:03:51.295794 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:51.376775 2774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:51.377017 2774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:51.377055 2774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:51.378180 2774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:51.378248 2774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:51.378293 2774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:03:51.379714 2774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40575
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:51.379974 2774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:51.380223 2774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:51.384923 2782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:51.386443 2780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:51.386413 2774 server_base.cc:1034] running on GCE node
W20260926 13:03:51.386406 2779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:51.386718 2774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:51.386911 2774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:51.388075 2774 hybrid_clock.cc:648] HybridClock initialized: now 1790427831388068 us; error 41 us; skew 500 ppm
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:51.390641 2774 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:03:51.391108 2774 webserver.cc:460] Webserver started at http://127.2.86.3:36363/ using document root <none> and password file <none>
I20260926 13:03:51.391350 2774 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:51.391415 2774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:51.391559 2774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:51.392558 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "c03d3af9256b4edf83b365e81a98f249"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "e505683d160eb80daeb5d86538c6caf9"
server_key_iv: "7431499e76789411ebc1b6f899583c54"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.392968 2774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "c03d3af9256b4edf83b365e81a98f249"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "e505683d160eb80daeb5d86538c6caf9"
server_key_iv: "7431499e76789411ebc1b6f899583c54"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.394220 2774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:03:51.394919 2789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.395089 2774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:03:51.395185 2774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "c03d3af9256b4edf83b365e81a98f249"
format_stamp: "Formatted at 2026-09-26 13:03:51 on dist-test-slave-j4lq"
server_key: "e505683d160eb80daeb5d86538c6caf9"
server_key_iv: "7431499e76789411ebc1b6f899583c54"
server_key_version: "encryptionkey@0"
I20260926 13:03:51.395270 2774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:51.405335 2774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:51.407686 2774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:51.407869 2774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:51.408097 2774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:51.408406 2774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:51.408463 2774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.408504 2774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:51.408532 2774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:51.415061 2774 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:42591
I20260926 13:03:51.415131 2902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:42591 every 8 connection(s)
I20260926 13:03:51.415457 2774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:03:51.420488 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 2774
I20260926 13:03:51.420585 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:03:51.420758 2392 external_mini_cluster.cc:1428] Setting key cf2f42173c249227849ff24f12ece0d3
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:51.425110 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.416274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:42383 (local address 127.2.86.62:40575)
0926 13:03:51.416331 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:51.416333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:51.416558 (+ 225us) server_negotiation.cc:407] Connection header received
0926 13:03:51.417136 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:51.417137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:51.417149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:51.417213 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:51.418121 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.418624 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.419646 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.419838 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.419922 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.422537 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:51.422538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:51.422538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:51.422544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:51.423821 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.424304 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.424305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.424305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.424353 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.424563 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.424563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.424564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.424662 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:51.424712 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:51.424988 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 13:03:51.425048 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:03:51.425375 2905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.416249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40575 (local address 127.2.86.3:42383)
0926 13:03:51.416437 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:51.416471 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:51.417030 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:51.417286 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:51.417295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:51.417836 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:51.418020 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.418025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.418715 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.418717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.419537 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:51.419539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.419930 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.419931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:51.420053 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.420531 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:51.420541 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:51.422463 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:51.423892 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.423897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.423899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.424111 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.424410 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:51.424411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:51.424411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:51.424509 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:51.424714 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:51.424718 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:51.424841 (+ 123us) client_negotiation.cc:769] Sending connection context
0926 13:03:51.424973 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 13:03:51.425094 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260926 13:03:51.425875 2903 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40575
I20260926 13:03:51.425952 2903 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:51.426157 2903 heartbeater.cc:511] Master 127.2.86.62:40575 requested a full tablet report, sending...
I20260926 13:03:51.426559 2444 ts_manager.cc:194] Registered new tserver with Master: c03d3af9256b4edf83b365e81a98f249 (127.2.86.3:42591)
I20260926 13:03:51.426949 2444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:42383
I20260926 13:03:51.432979 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:51.455770 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.448551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.2.86.1:41621)
0926 13:03:51.448720 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:51.448725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:51.448735 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:03:51.448761 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:51.448763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:51.448776 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:51.448841 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:51.449410 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.450107 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.450801 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.451117 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.451169 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.453378 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:51.453388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:51.453388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:51.453396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:51.454778 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.455083 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.455086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.455086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.455138 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.455302 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.455303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.455303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.455432 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:51.455547 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:51.455590 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 13:03:51.455648 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260926 13:03:51.456126 2549 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45106
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:51.477463 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:51.470964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45112 (local address 127.2.86.1:41621)
0926 13:03:51.471026 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:51.471028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:51.471075 (+ 47us) server_negotiation.cc:407] Connection header received
0926 13:03:51.471124 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:51.471125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:51.471136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:51.471184 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:51.471682 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.472164 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.472830 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:51.473068 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:51.473138 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:51.475274 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:51.475276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:51.475276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:51.475282 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:51.476490 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.476850 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.476851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.476851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.476941 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:51.477103 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:51.477104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:51.477104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:51.477191 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:51.477279 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:51.477340 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 13:03:51.477400 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:03:51.477981 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 2506
I20260926 13:03:51.483343 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 2640
I20260926 13:03:51.488778 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 2774
I20260926 13:03:51.493115 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 2412
2026-09-26T13:03:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4993 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2917](info): set up 2 sockets
Sep 26 13:03:51 dist-test-slave-j4lq krb5kdc[2917](info): commencing operation
krb5kdc: starting...
W20260926 13:03:52.528435 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:03:52 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:03:52.640106 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:35873
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:39371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:35873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:52.717408 2930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:52.717636 2930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:52.717672 2930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:52.718773 2930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:03:52.718842 2930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:52.718871 2930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:52.718902 2930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:03:52.718923 2930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:03:52.720290 2930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:35873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:35873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:52.720543 2930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:52.720836 2930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:52.726464 2936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:52.726576 2938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:52.726487 2935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:52.726516 2930 server_base.cc:1034] running on GCE node
I20260926 13:03:52.727010 2930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:52.727389 2930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:52.728564 2930 hybrid_clock.cc:648] HybridClock initialized: now 1790427832728545 us; error 35 us; skew 500 ppm
Sep 26 13:03:52 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:52.743686 2930 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:03:52.744223 2930 webserver.cc:460] Webserver started at http://127.2.86.62:43793/ using document root <none> and password file <none>
I20260926 13:03:52.744423 2930 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:52.744465 2930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:52.744557 2930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:52.745692 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "2e54d19f430b4606ae7e50f393dcdc49"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "7b9d9792de5fe06d615173ba67795649"
server_key_iv: "57568f5bf924e28432534b8a7207c495"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.746004 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "2e54d19f430b4606ae7e50f393dcdc49"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "7b9d9792de5fe06d615173ba67795649"
server_key_iv: "57568f5bf924e28432534b8a7207c495"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.747251 2930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:03:52.747916 2945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:52.748085 2930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:03:52.748190 2930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "2e54d19f430b4606ae7e50f393dcdc49"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "7b9d9792de5fe06d615173ba67795649"
server_key_iv: "57568f5bf924e28432534b8a7207c495"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.748267 2930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:52.769008 2930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:52.770797 2930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:52.770962 2930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:52.774989 2930 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:35873
I20260926 13:03:52.775024 2997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:35873 every 8 connection(s)
I20260926 13:03:52.775374 2930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:03:52.776170 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:52.778388 2998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49: Bootstrap starting.
I20260926 13:03:52.778942 2998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:52.779238 2998 log.cc:824] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:52.779881 2998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49: No bootstrap required, opened a new log
I20260926 13:03:52.781433 2998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } }
I20260926 13:03:52.781558 2998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:52.781580 2998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e54d19f430b4606ae7e50f393dcdc49, State: Initialized, Role: FOLLOWER
I20260926 13:03:52.781757 2998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } }
I20260926 13:03:52.781817 2998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:03:52.781837 2998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:03:52.781903 2998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:52.782372 2998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } }
I20260926 13:03:52.782476 2998 leader_election.cc:302] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e54d19f430b4606ae7e50f393dcdc49; no voters:
I20260926 13:03:52.782645 2998 leader_election.cc:288] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:03:52.782716 3001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:52.782927 2998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:03:52.783116 3001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [term 1 LEADER]: Becoming Leader. State: Replica: 2e54d19f430b4606ae7e50f393dcdc49, State: Running, Role: LEADER
I20260926 13:03:52.783244 3001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } }
I20260926 13:03:52.783605 3003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e54d19f430b4606ae7e50f393dcdc49. Latest consensus state: current_term: 1 leader_uuid: "2e54d19f430b4606ae7e50f393dcdc49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } } }
I20260926 13:03:52.783694 3003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:52.783614 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e54d19f430b4606ae7e50f393dcdc49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54d19f430b4606ae7e50f393dcdc49" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 35873 } } }
I20260926 13:03:52.783793 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49 [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:52.783914 3010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:03:52.784389 3010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:03:52.784737 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 2930
I20260926 13:03:52.784821 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:03:52.785003 2392 external_mini_cluster.cc:1428] Setting key 51b7bdb8f475ca474b7b59904d537c63
I20260926 13:03:52.786438 3010 catalog_manager.cc:1347] Generated new cluster ID: aa7dfd8ce5ca49a1b02c75351dbfad5a
I20260926 13:03:52.786489 3010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:03:52 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:52.796139 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:52.785429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.2.86.62:35873)
0926 13:03:52.785621 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:52.785629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:52.785816 (+ 187us) server_negotiation.cc:407] Connection header received
0926 13:03:52.785926 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:52.785931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:52.785972 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:52.786090 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:52.786814 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.788160 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.789243 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.789808 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.789899 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:52.792577 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:52.792581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:52.792582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:52.792593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:52.794565 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:52.794941 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:52.794944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:52.794945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:52.795054 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:52.795257 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:52.795258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:52.795258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:52.795553 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:52.795722 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:52.795792 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 13:03:52.795927 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260926 13:03:52.798023 3010 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:03:52.798519 3010 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:03:52.802019 3010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e54d19f430b4606ae7e50f393dcdc49: Generated new TSK 0
I20260926 13:03:52.802189 3010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:03:52.828016 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:39371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:52.907229 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:52.907449 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:52.907490 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:52.908627 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:52.908691 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:52.908735 3024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:03:52.910228 3024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:52.910475 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:52.910689 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:52.915575 3032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:52.915709 3024 server_base.cc:1034] running on GCE node
W20260926 13:03:52.915644 3029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:52.915572 3030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:52.916067 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:52.916339 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:52.917493 3024 hybrid_clock.cc:648] HybridClock initialized: now 1790427832917495 us; error 38 us; skew 500 ppm
Sep 26 13:03:52 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:52.919914 3024 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:03:52.920338 3024 webserver.cc:460] Webserver started at http://127.2.86.1:45111/ using document root <none> and password file <none>
I20260926 13:03:52.920511 3024 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:52.920552 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:52.920634 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:52.921657 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "ca1d0329951c4e3b82297b4f61805170"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "13bf4c26bb87593d41b01fac14cdf455"
server_key_iv: "3613664538778e5ec8802fcbf52fca3d"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.921952 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "ca1d0329951c4e3b82297b4f61805170"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "13bf4c26bb87593d41b01fac14cdf455"
server_key_iv: "3613664538778e5ec8802fcbf52fca3d"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.923264 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:03:52.923897 3039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:52.924088 3024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:52.924168 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "ca1d0329951c4e3b82297b4f61805170"
format_stamp: "Formatted at 2026-09-26 13:03:52 on dist-test-slave-j4lq"
server_key: "13bf4c26bb87593d41b01fac14cdf455"
server_key_iv: "3613664538778e5ec8802fcbf52fca3d"
server_key_version: "encryptionkey@0"
I20260926 13:03:52.924218 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:52.935770 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:52.938236 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:52.938346 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:52.938524 3024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:52.938766 3024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:52.938794 3024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:52.938817 3024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:52.938828 3024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:52.944360 3024 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:42799
I20260926 13:03:52.944471 3152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:42799 every 8 connection(s)
I20260926 13:03:52.944765 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:03:52 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:52.952521 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3024
I20260926 13:03:52.952597 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:03:52.952781 2392 external_mini_cluster.cc:1428] Setting key 3995660c91ad73176b9a35863ee7de7f
I20260926 13:03:52.954129 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:52.945511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:35703 (local address 127.2.86.62:35873)
0926 13:03:52.945579 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:52.945581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:52.945792 (+ 211us) server_negotiation.cc:407] Connection header received
0926 13:03:52.946697 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:52.946698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:52.946711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:52.946761 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:52.947323 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.947815 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.948766 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.949034 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.949086 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:52.951146 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:52.951148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:52.951149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:52.951156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:52.952551 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:52.953103 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:52.953104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:52.953105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:52.953171 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:52.953450 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:52.953451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:52.953451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:52.953591 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:52.953673 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:52.953950 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 13:03:52.954038 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:03:52.954326 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:52.945513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:35873 (local address 127.2.86.1:35703)
0926 13:03:52.945672 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:52.945707 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:52.946606 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:52.946823 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:52.946826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:52.947066 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:52.947251 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:52.947255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.947900 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.947900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:52.948692 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:52.948694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:52.949101 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:52.949102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:52.949202 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:52.949634 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:52.949643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:52.951051 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:52.952660 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:52.952667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:52.952669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:52.953002 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:52.953236 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:52.953237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:52.953237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:52.953367 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:52.953666 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:52.953672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:52.953815 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 13:03:52.953929 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 13:03:52.954084 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260926 13:03:52.954802 3153 heartbeater.cc:347] Connected to a master server at 127.2.86.62:35873
I20260926 13:03:52.954909 3153 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:52.955082 3153 heartbeater.cc:511] Master 127.2.86.62:35873 requested a full tablet report, sending...
I20260926 13:03:52.955633 2962 ts_manager.cc:194] Registered new tserver with Master: ca1d0329951c4e3b82297b4f61805170 (127.2.86.1:42799)
I20260926 13:03:52.956238 2962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:35703
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:03:52.975940 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:39371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:53.055342 3158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:53.055555 3158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:53.055593 3158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:53.056704 3158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:53.056800 3158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:53.056852 3158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:03:53.058326 3158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:53.058595 3158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:53.058813 3158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:53.063628 3164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:53.063637 3158 server_base.cc:1034] running on GCE node
W20260926 13:03:53.063606 3166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:53.063606 3163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:53.064028 3158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:53.064299 3158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:53.065436 3158 hybrid_clock.cc:648] HybridClock initialized: now 1790427833065425 us; error 29 us; skew 500 ppm
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:53.067899 3158 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:03:53.068287 3158 webserver.cc:460] Webserver started at http://127.2.86.2:36227/ using document root <none> and password file <none>
I20260926 13:03:53.068511 3158 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:53.068553 3158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:53.068710 3158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:53.069761 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "ca1518fa76cf4b08a29a97e532c1b02c"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "b2b54303a5b76c6a0201ea19d04f2d75"
server_key_iv: "308e574a670932d8583eee45119b1258"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.070170 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "ca1518fa76cf4b08a29a97e532c1b02c"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "b2b54303a5b76c6a0201ea19d04f2d75"
server_key_iv: "308e574a670932d8583eee45119b1258"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.071398 3158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:03:53.072059 3173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.072217 3158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:53.072309 3158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "ca1518fa76cf4b08a29a97e532c1b02c"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "b2b54303a5b76c6a0201ea19d04f2d75"
server_key_iv: "308e574a670932d8583eee45119b1258"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.072391 3158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:53.093176 3158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:53.095397 3158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:53.095556 3158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:53.095773 3158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:53.096072 3158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:53.096125 3158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.096166 3158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:53.096194 3158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.101621 3158 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:43827
I20260926 13:03:53.101673 3286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:43827 every 8 connection(s)
I20260926 13:03:53.102000 3158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:53.110422 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3158
I20260926 13:03:53.110520 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:03:53.110729 2392 external_mini_cluster.cc:1428] Setting key 989f69298f9d4640282bc033fa65075f
I20260926 13:03:53.111241 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.102785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:60703 (local address 127.2.86.62:35873)
0926 13:03:53.102875 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.102877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.103070 (+ 193us) server_negotiation.cc:407] Connection header received
0926 13:03:53.104019 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.104021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.104033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.104085 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.104698 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.105279 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.106253 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.106476 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.106564 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.108525 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.108527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.108527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.108535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.109787 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.110183 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.110184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.110185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.110244 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.110507 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.110508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.110508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.110685 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.110748 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.111067 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.111155 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:03:53.111480 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.102752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:35873 (local address 127.2.86.2:60703)
0926 13:03:53.102935 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.102969 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.103909 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.104172 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.104175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.104420 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:53.104600 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.104604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.105372 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.105373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.106158 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.106160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.106572 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.106573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.106694 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.107113 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:53.107121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:53.108428 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:53.109887 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:53.109894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:53.109896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:53.110107 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:53.110340 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:53.110341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:53.110341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:53.110440 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:53.110738 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:53.110744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:53.110925 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.111046 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.111193 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260926 13:03:53.112044 3287 heartbeater.cc:347] Connected to a master server at 127.2.86.62:35873
I20260926 13:03:53.112145 3287 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:53.112327 3287 heartbeater.cc:511] Master 127.2.86.62:35873 requested a full tablet report, sending...
I20260926 13:03:53.112725 2962 ts_manager.cc:194] Registered new tserver with Master: ca1518fa76cf4b08a29a97e532c1b02c (127.2.86.2:43827)
I20260926 13:03:53.113159 2962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:60703
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:03:53.135025 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:39371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:53.212718 3292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:53.212944 3292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:53.212980 3292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:53.214057 3292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:53.214118 3292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:53.214159 3292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:03:53.215629 3292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:35873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:53.215888 3292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:53.216094 3292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:53.220647 3300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:53.220769 3292 server_base.cc:1034] running on GCE node
W20260926 13:03:53.220666 3297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:53.220804 3298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:53.221141 3292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:53.221383 3292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:53.222515 3292 hybrid_clock.cc:648] HybridClock initialized: now 1790427833222504 us; error 39 us; skew 500 ppm
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:53.224838 3292 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:03:53.225306 3292 webserver.cc:460] Webserver started at http://127.2.86.3:36233/ using document root <none> and password file <none>
I20260926 13:03:53.225528 3292 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:53.225572 3292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:53.225713 3292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:53.226656 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "9b9511a86a724668bc55c6bdc3791208"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "d2c6ce19e65e155ee012f1c898960ff1"
server_key_iv: "f8b9a7146e2e1be34ce6a2693f1aaea2"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.227010 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "9b9511a86a724668bc55c6bdc3791208"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "d2c6ce19e65e155ee012f1c898960ff1"
server_key_iv: "f8b9a7146e2e1be34ce6a2693f1aaea2"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.228281 3292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:03:53.228955 3307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.229169 3292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:53.229247 3292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "9b9511a86a724668bc55c6bdc3791208"
format_stamp: "Formatted at 2026-09-26 13:03:53 on dist-test-slave-j4lq"
server_key: "d2c6ce19e65e155ee012f1c898960ff1"
server_key_iv: "f8b9a7146e2e1be34ce6a2693f1aaea2"
server_key_version: "encryptionkey@0"
I20260926 13:03:53.229295 3292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:53.237349 3292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:53.239003 3292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:53.239106 3292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:53.239305 3292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:03:53.239554 3292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:53.239584 3292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.239605 3292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:53.239619 3292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:53.244974 3292 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:36413
I20260926 13:03:53.245033 3420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:36413 every 8 connection(s)
I20260926 13:03:53.245314 3292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:03:53.249455 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3292
I20260926 13:03:53.249560 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:03:53.249727 2392 external_mini_cluster.cc:1428] Setting key f8ece433cc743f74ca38dbe2b2bc25db
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:53.254117 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.246017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:54961 (local address 127.2.86.62:35873)
0926 13:03:53.246091 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.246093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.246263 (+ 170us) server_negotiation.cc:407] Connection header received
0926 13:03:53.247096 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.247097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.247106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.247157 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.247694 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.248190 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.249207 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.249394 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.249478 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.251675 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.251677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.251677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.251684 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.252772 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.253175 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.253176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.253176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.253237 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.253493 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.253494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.253494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.253602 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.253673 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.253951 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.254027 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:03:53.254299 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.245978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:35873 (local address 127.2.86.3:54961)
0926 13:03:53.246147 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.246179 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.247014 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.247218 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.247221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.247456 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:53.247624 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.247628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.248266 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.248267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.249140 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.249142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.249489 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.249490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.249664 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.250177 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:53.250186 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:53.251602 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:53.252888 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:53.252894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:53.252896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:53.253098 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:53.253313 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:53.253314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:53.253315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:53.253421 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:53.253664 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:53.253670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:53.253819 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.253926 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.254071 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260926 13:03:53.254737 3421 heartbeater.cc:347] Connected to a master server at 127.2.86.62:35873
I20260926 13:03:53.254834 3421 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:53.254997 3421 heartbeater.cc:511] Master 127.2.86.62:35873 requested a full tablet report, sending...
I20260926 13:03:53.255344 2962 ts_manager.cc:194] Registered new tserver with Master: 9b9511a86a724668bc55c6bdc3791208 (127.2.86.3:36413)
I20260926 13:03:53.255697 2962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:54961
I20260926 13:03:53.261327 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:03:53.268335 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.262813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.2.86.62:35873)
0926 13:03:53.262885 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.262888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.262989 (+ 101us) server_negotiation.cc:407] Connection header received
0926 13:03:53.263061 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.263062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.263071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.263124 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.263654 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.264120 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.264822 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.265062 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.265144 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.266170 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.266171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.266171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.266178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.267257 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.267587 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.267588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.267588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.267640 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.267833 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.267834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.267834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.267970 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.268069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.268174 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.268244 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:03:53.270175 2962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:03:53.271566 2962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:53.277551 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42799 (local address 127.0.0.1:53784)
0926 13:03:53.273761 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.273801 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.273883 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.274283 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.274284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.274286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:53.274324 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.274325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275422 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.275423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.276451 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.276454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.277268 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.277269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.277349 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.277363 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.277435 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.277481 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":55,"thread_start_us":45,"threads_started":1}
I20260926 13:03:53.277599 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.2.86.1:42799)
0926 13:03:53.274052 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.274056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.274063 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:03:53.274111 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.274114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.274128 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.274201 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:53.274410 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275323 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.276535 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.277188 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.277262 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.277313 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.277462 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.277503 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":91,"threads_started":1}
I20260926 13:03:53.277915 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.2.86.3:36413)
0926 13:03:53.273962 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.273966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.273978 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:03:53.274016 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.274017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.274033 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.274125 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:53.274286 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275436 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.276810 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.277517 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.277580 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.277628 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.277804 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.277841 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260926 13:03:53.277930 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:36413 (local address 127.0.0.1:43350)
0926 13:03:53.273896 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.273906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.273945 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.274143 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.274145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.274148 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:53.274210 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.274212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275527 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.275528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.276734 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.276736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.277665 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.277666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.277728 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.277732 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.277788 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.277830 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":19,"thread_start_us":48,"threads_started":1}
I20260926 13:03:53.278267 3087 tablet_service.cc:1467] Processing CreateTablet for tablet 88d1c73be8e44bda9e3cd020dce81165 (DEFAULT_TABLE table=test-table [id=88f57ae82b5d48ac85c91b1c056be8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:53.278579 3087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d1c73be8e44bda9e3cd020dce81165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:53.278640 3355 tablet_service.cc:1467] Processing CreateTablet for tablet 88d1c73be8e44bda9e3cd020dce81165 (DEFAULT_TABLE table=test-table [id=88f57ae82b5d48ac85c91b1c056be8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:53.278944 3355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d1c73be8e44bda9e3cd020dce81165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:53.279037 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:43827 (local address 127.0.0.1:41154)
0926 13:03:53.274078 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.274087 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.274134 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.274434 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.274435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.274438 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:53.274476 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.274478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275690 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.275690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.277060 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.277062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.278600 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.278602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.278726 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.278730 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.278864 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.278911 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260926 13:03:53.279376 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.273847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.2.86.2:43827)
0926 13:03:53.274025 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.274028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.274190 (+ 162us) server_negotiation.cc:407] Connection header received
0926 13:03:53.274232 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.274233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.274264 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.274336 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:53.274560 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.275597 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.277138 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.278374 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.278519 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.278574 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.279250 (+ 676us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.279293 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260926 13:03:53.280181 3221 tablet_service.cc:1467] Processing CreateTablet for tablet 88d1c73be8e44bda9e3cd020dce81165 (DEFAULT_TABLE table=test-table [id=88f57ae82b5d48ac85c91b1c056be8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:53.280483 3221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d1c73be8e44bda9e3cd020dce81165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:53.281311 3435 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: Bootstrap starting.
I20260926 13:03:53.282078 3436 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: Bootstrap starting.
I20260926 13:03:53.282248 3435 tablet_bootstrap.cc:654] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:53.282719 3436 tablet_bootstrap.cc:654] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:53.282912 3435 log.cc:824] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:53.283684 3436 log.cc:824] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:53.283685 3437 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Bootstrap starting.
I20260926 13:03:53.284365 3437 tablet_bootstrap.cc:654] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:53.284448 3436 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: No bootstrap required, opened a new log
I20260926 13:03:53.284511 3435 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: No bootstrap required, opened a new log
I20260926 13:03:53.284516 3436 ts_tablet_manager.cc:1397] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:03:53.284583 3435 ts_tablet_manager.cc:1397] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:03:53.285027 3437 log.cc:824] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:53.285756 3437 tablet_bootstrap.cc:492] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: No bootstrap required, opened a new log
I20260926 13:03:53.285822 3437 ts_tablet_manager.cc:1397] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:53.286214 3436 raft_consensus.cc:348] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.286366 3436 raft_consensus.cc:374] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:53.286407 3436 raft_consensus.cc:729] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9511a86a724668bc55c6bdc3791208, State: Initialized, Role: FOLLOWER
I20260926 13:03:53.286511 3436 consensus_queue.cc:260] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.286620 3435 raft_consensus.cc:348] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.286782 3435 raft_consensus.cc:374] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:53.286777 3421 heartbeater.cc:503] Master 127.2.86.62:35873 was elected leader, sending a full tablet report...
I20260926 13:03:53.286815 3435 raft_consensus.cc:729] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1d0329951c4e3b82297b4f61805170, State: Initialized, Role: FOLLOWER
I20260926 13:03:53.286926 3435 consensus_queue.cc:260] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.286778 3436 ts_tablet_manager.cc:1428] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:03:53.287163 3435 ts_tablet_manager.cc:1428] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:03:53.287297 3153 heartbeater.cc:503] Master 127.2.86.62:35873 was elected leader, sending a full tablet report...
I20260926 13:03:53.287748 3437 raft_consensus.cc:348] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.287926 3437 raft_consensus.cc:374] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:53.287963 3437 raft_consensus.cc:729] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1518fa76cf4b08a29a97e532c1b02c, State: Initialized, Role: FOLLOWER
I20260926 13:03:53.288055 3437 consensus_queue.cc:260] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.288265 3437 ts_tablet_manager.cc:1428] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:03:53.288296 3287 heartbeater.cc:503] Master 127.2.86.62:35873 was elected leader, sending a full tablet report...
I20260926 13:03:53.294183 3442 raft_consensus.cc:482] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:53.294248 3442 raft_consensus.cc:504] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.294626 3442 leader_election.cc:288] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1518fa76cf4b08a29a97e532c1b02c (127.2.86.2:43827), 9b9511a86a724668bc55c6bdc3791208 (127.2.86.3:36413)
I20260926 13:03:53.297377 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.294891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:32789 (local address 127.2.86.3:36413)
0926 13:03:53.294932 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.294934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.295021 (+ 87us) server_negotiation.cc:407] Connection header received
0926 13:03:53.295070 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.295071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.295080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.295133 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:53.295245 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.295685 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.296607 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.297066 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.297120 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.297127 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.297279 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.297318 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:03:53.297402 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.294777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:36413 (local address 127.2.86.1:32789)
0926 13:03:53.294957 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.294967 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.295021 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.295133 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.295134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.295135 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:53.295191 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.295192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.295767 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.295767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.296525 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.296527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.297138 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.297139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.297210 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.297214 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.297266 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.297300 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":10,"thread_start_us":71,"threads_started":1}
I20260926 13:03:53.297567 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.294777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:43827 (local address 127.2.86.1:40871)
0926 13:03:53.294826 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:53.294838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:53.294874 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:53.295017 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:53.295018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:53.295020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:53.295083 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.295085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.295703 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.295704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.296587 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:53.296590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.297195 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.297196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:53.297279 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.297283 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:53.297337 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:03:53.297382 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":23}
I20260926 13:03:53.297573 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.294830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:40871 (local address 127.2.86.2:43827)
0926 13:03:53.294867 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.294869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.294878 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:03:53.294927 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.294928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.294939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.295010 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:53.295133 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.295630 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.296674 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.297112 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.297182 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.297190 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.297354 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.297387 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:03:53.297724 3375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d1c73be8e44bda9e3cd020dce81165" candidate_uuid: "ca1d0329951c4e3b82297b4f61805170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9511a86a724668bc55c6bdc3791208" is_pre_election: true
I20260926 13:03:53.297892 3375 raft_consensus.cc:2454] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1d0329951c4e3b82297b4f61805170 in term 0.
I20260926 13:03:53.297930 3241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d1c73be8e44bda9e3cd020dce81165" candidate_uuid: "ca1d0329951c4e3b82297b4f61805170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" is_pre_election: true
I20260926 13:03:53.298043 3241 raft_consensus.cc:2454] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1d0329951c4e3b82297b4f61805170 in term 0.
I20260926 13:03:53.298125 3043 leader_election.cc:302] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9511a86a724668bc55c6bdc3791208, ca1d0329951c4e3b82297b4f61805170; no voters:
I20260926 13:03:53.298246 3442 raft_consensus.cc:2781] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:53.298307 3442 raft_consensus.cc:482] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:53.298326 3442 raft_consensus.cc:3037] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:53.298894 3442 raft_consensus.cc:504] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.299023 3442 leader_election.cc:288] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [CANDIDATE]: Term 1 election: Requested vote from peers ca1518fa76cf4b08a29a97e532c1b02c (127.2.86.2:43827), 9b9511a86a724668bc55c6bdc3791208 (127.2.86.3:36413)
I20260926 13:03:53.299163 3375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d1c73be8e44bda9e3cd020dce81165" candidate_uuid: "ca1d0329951c4e3b82297b4f61805170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b9511a86a724668bc55c6bdc3791208"
I20260926 13:03:53.299163 3241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d1c73be8e44bda9e3cd020dce81165" candidate_uuid: "ca1d0329951c4e3b82297b4f61805170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1518fa76cf4b08a29a97e532c1b02c"
I20260926 13:03:53.299230 3241 raft_consensus.cc:3037] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:53.299230 3375 raft_consensus.cc:3037] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:53.300439 3241 raft_consensus.cc:2454] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1d0329951c4e3b82297b4f61805170 in term 1.
I20260926 13:03:53.300439 3375 raft_consensus.cc:2454] T 88d1c73be8e44bda9e3cd020dce81165 P 9b9511a86a724668bc55c6bdc3791208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1d0329951c4e3b82297b4f61805170 in term 1.
I20260926 13:03:53.300606 3043 leader_election.cc:302] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b9511a86a724668bc55c6bdc3791208, ca1d0329951c4e3b82297b4f61805170; no voters:
I20260926 13:03:53.300719 3442 raft_consensus.cc:2781] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:53.300855 3442 raft_consensus.cc:686] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [term 1 LEADER]: Becoming Leader. State: Replica: ca1d0329951c4e3b82297b4f61805170, State: Running, Role: LEADER
I20260926 13:03:53.300962 3442 consensus_queue.cc:237] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } }
I20260926 13:03:53.301750 2961 catalog_manager.cc:5495] T 88d1c73be8e44bda9e3cd020dce81165 P ca1d0329951c4e3b82297b4f61805170 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca1d0329951c4e3b82297b4f61805170 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "ca1d0329951c4e3b82297b4f61805170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d0329951c4e3b82297b4f61805170" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca1518fa76cf4b08a29a97e532c1b02c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b9511a86a724668bc55c6bdc3791208" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36413 } health_report { overall_health: UNKNOWN } } }
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:53.316212 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.309444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.2.86.1:42799)
0926 13:03:53.309500 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.309502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.309620 (+ 118us) server_negotiation.cc:407] Connection header received
0926 13:03:53.309656 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.309656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.309665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.309718 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.310191 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.310630 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.311279 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.311456 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.311503 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.313857 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.313864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.313864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.313870 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.315291 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.315596 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.315598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.315598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.315661 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.315819 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.315820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.315820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.315945 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.316057 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.316093 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.316153 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:53.338551 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.331870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.2.86.1:42799)
0926 13:03:53.331940 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.331943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.331995 (+ 52us) server_negotiation.cc:407] Connection header received
0926 13:03:53.332066 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.332067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.332076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.332125 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.332652 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.333244 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.333996 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.334179 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.334229 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.336352 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.336353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.336353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.336359 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.337671 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.337976 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.337976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.337977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.338021 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.338176 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.338176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.338176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.338255 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.338325 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.338438 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.338491 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260926 13:03:53.352509 3288 tablet.cc:2369] T 88d1c73be8e44bda9e3cd020dce81165 P ca1518fa76cf4b08a29a97e532c1b02c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[2917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:53.359817 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:53.353480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.2.86.1:42799)
0926 13:03:53.353543 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:53.353545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:53.353676 (+ 131us) server_negotiation.cc:407] Connection header received
0926 13:03:53.353721 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:53.353721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:53.353730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:53.353779 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:53.354272 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.354726 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.355404 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:53.355574 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:53.355621 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:53.357669 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:53.357670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:53.357670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:53.357676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:53.359002 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.359268 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.359268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.359269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.359314 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:53.359448 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:53.359448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:53.359449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:53.359522 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:53.359588 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:53.359672 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 13:03:53.359723 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 13:03:53.360287 3067 tablet_service.cc:2908] Rejecting scan request for tablet 88d1c73be8e44bda9e3cd020dce81165: Uninitialized: clean time has not yet been initialized
I20260926 13:03:53.361104 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3024
I20260926 13:03:53.366516 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3158
I20260926 13:03:53.370684 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3292
I20260926 13:03:53.374928 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 2930
2026-09-26T13:03:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1882 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[3459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[3459](info): set up 2 sockets
Sep 26 13:03:53 dist-test-slave-j4lq krb5kdc[3459](info): commencing operation
krb5kdc: starting...
W20260926 13:03:54.392467 2392 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:03:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:03:54.507218 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33523
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:54.585526 3472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:54.585722 3472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:54.585744 3472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:54.586796 3472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:03:54.586829 3472 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:03:54.586843 3472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:54.586855 3472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:54.586867 3472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:03:54.586880 3472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:03:54.588245 3472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33523
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:54.588442 3472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:54.588613 3472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:54.593254 3478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:54.593318 3477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:54.593346 3480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:54.593439 3472 server_base.cc:1034] running on GCE node
I20260926 13:03:54.593688 3472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:54.594005 3472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:54.595158 3472 hybrid_clock.cc:648] HybridClock initialized: now 1790427834595144 us; error 33 us; skew 500 ppm
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:54.597688 3472 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:03:54.598109 3472 webserver.cc:460] Webserver started at http://127.2.86.62:42171/ using document root <none> and password file <none>
I20260926 13:03:54.598325 3472 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:54.598394 3472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:54.598543 3472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:54.599547 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "a4c26289b3544889ab3bacc8eff4281a"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "80d94ff1d790100272fc4f09f817cdca"
server_key_iv: "65d6e1422f686af658894c1d2539eca2"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.599907 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "a4c26289b3544889ab3bacc8eff4281a"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "80d94ff1d790100272fc4f09f817cdca"
server_key_iv: "65d6e1422f686af658894c1d2539eca2"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.601221 3472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 13:03:54.601967 3487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:54.602145 3472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:54.602241 3472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "a4c26289b3544889ab3bacc8eff4281a"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "80d94ff1d790100272fc4f09f817cdca"
server_key_iv: "65d6e1422f686af658894c1d2539eca2"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.602316 3472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:54.630193 3472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:54.632043 3472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:54.632213 3472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:54.636909 3472 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33523
I20260926 13:03:54.636968 3549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33523 every 8 connection(s)
I20260926 13:03:54.637266 3472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:03:54.637869 3550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:54.640257 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Bootstrap starting.
I20260926 13:03:54.640841 3550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:54.641181 3550 log.cc:824] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:54.641832 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: No bootstrap required, opened a new log
I20260926 13:03:54.642077 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3472
I20260926 13:03:54.642151 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:03:54.642310 2392 external_mini_cluster.cc:1428] Setting key aaf365dbfdba3a2858d66523d23de7e0
I20260926 13:03:54.644246 3550 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } }
I20260926 13:03:54.644398 3550 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:54.644476 3550 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c26289b3544889ab3bacc8eff4281a, State: Initialized, Role: FOLLOWER
I20260926 13:03:54.644647 3550 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } }
I20260926 13:03:54.644743 3550 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:03:54.644786 3550 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:03:54.644842 3550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:54.645527 3550 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } }
I20260926 13:03:54.645632 3550 leader_election.cc:302] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4c26289b3544889ab3bacc8eff4281a; no voters:
I20260926 13:03:54.645777 3550 leader_election.cc:288] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:03:54.645833 3555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:54.645975 3555 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [term 1 LEADER]: Becoming Leader. State: Replica: a4c26289b3544889ab3bacc8eff4281a, State: Running, Role: LEADER
I20260926 13:03:54.645996 3550 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:03:54.646068 3555 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } }
I20260926 13:03:54.646405 3555 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4c26289b3544889ab3bacc8eff4281a. Latest consensus state: current_term: 1 leader_uuid: "a4c26289b3544889ab3bacc8eff4281a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } } }
I20260926 13:03:54.646476 3555 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:54.647591 3557 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4c26289b3544889ab3bacc8eff4281a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c26289b3544889ab3bacc8eff4281a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33523 } } }
I20260926 13:03:54.647667 3557 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a [sys.catalog]: This master's current role is: LEADER
W20260926 13:03:54.647796 3570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:03:54.647838 3570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:03:54.647900 3562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:03:54.648068 3562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:03:54.649662 3562 catalog_manager.cc:1347] Generated new cluster ID: 8f57410ac83b46f3a3129a73d27a4d13
I20260926 13:03:54.649715 3562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:54.653584 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:54.642846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.2.86.62:33523)
0926 13:03:54.643029 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:54.643037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:54.643075 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:03:54.643924 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:54.643932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:54.643979 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:54.644120 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:54.644848 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.646250 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.647248 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.647874 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.648000 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:54.650608 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:54.650611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:54.650611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:54.650620 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:54.652068 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.652448 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.652451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.652451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.652528 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.652707 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.652707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.652708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.652982 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:54.653137 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:54.653235 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:03:54.653389 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260926 13:03:54.676380 3562 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:03:54.676985 3562 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:03:54.682576 3562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Generated new TSK 0
I20260926 13:03:54.682749 3562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:03:54.715281 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33523
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:54.811517 3576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:54.811702 3576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:54.811722 3576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:54.812793 3576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:54.812831 3576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:54.812850 3576 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:54.812912 3576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:03:54.814205 3576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:54.814379 3576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:54.814606 3576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:54.819420 3581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:54.819416 3582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:54.819432 3576 server_base.cc:1034] running on GCE node
W20260926 13:03:54.819415 3584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:54.819965 3576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:54.820210 3576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:54.821367 3576 hybrid_clock.cc:648] HybridClock initialized: now 1790427834821360 us; error 35 us; skew 500 ppm
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:54.823817 3576 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:03:54.824237 3576 webserver.cc:460] Webserver started at http://127.2.86.1:35739/ using document root <none> and password file <none>
I20260926 13:03:54.824414 3576 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:54.824452 3576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:54.824535 3576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:54.825536 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "bfe6bbe1c5414c4cbedf3974a2af6002"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "2f5b8b55d1c9bc7eff6584fc517e535d"
server_key_iv: "08d6f247fd2d5d0715ef9173728810df"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.825825 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "bfe6bbe1c5414c4cbedf3974a2af6002"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "2f5b8b55d1c9bc7eff6584fc517e535d"
server_key_iv: "08d6f247fd2d5d0715ef9173728810df"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.826979 3576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:03:54.827570 3591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:54.827733 3576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:54.827827 3576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "bfe6bbe1c5414c4cbedf3974a2af6002"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "2f5b8b55d1c9bc7eff6584fc517e535d"
server_key_iv: "08d6f247fd2d5d0715ef9173728810df"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.827883 3576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:54.850809 3576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:54.852523 3576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:54.852627 3576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:54.853124 3576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:54.853161 3576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:54.853183 3576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:54.853230 3576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:54.860024 3576 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:40515
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:54.860090 3706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:40515 every 8 connection(s)
I20260926 13:03:54.860410 3576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:03:54.862865 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:54.853549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:37619 (local address 127.2.86.62:33523)
0926 13:03:54.853621 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:54.853624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:54.853803 (+ 179us) server_negotiation.cc:407] Connection header received
0926 13:03:54.854255 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:54.854256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:54.854269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:54.854353 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:54.855003 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.855831 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.857406 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.857770 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.857859 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:54.860615 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:54.860617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:54.860617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:54.860625 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:54.861803 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.862146 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.862147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.862147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.862195 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.862392 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.862393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.862393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.862492 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:54.862538 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:54.862743 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 13:03:54.862807 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:03:54.863044 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:54.853525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.1:37619)
0926 13:03:54.853671 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:54.853708 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:54.854156 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:54.854409 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:54.854412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:54.854626 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:54.854884 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:54.854891 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.855934 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.855935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:54.857311 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:54.857314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.857892 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.857893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:54.858099 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:54.858645 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:54.858652 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:54.860516 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:54.861878 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:54.861883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:54.861884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:54.862083 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:54.862250 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:54.862250 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:54.862251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:54.862340 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:54.862543 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:54.862547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:54.862643 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 13:03:54.862716 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:03:54.862820 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260926 13:03:54.863498 3707 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33523
I20260926 13:03:54.863576 3707 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:54.863754 3707 heartbeater.cc:511] Master 127.2.86.62:33523 requested a full tablet report, sending...
I20260926 13:03:54.864250 3504 ts_manager.cc:194] Registered new tserver with Master: bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515)
I20260926 13:03:54.865208 3504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:37619
I20260926 13:03:54.869441 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:54.864582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:51733 (local address 127.2.86.62:33523)
0926 13:03:54.864622 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:54.864624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:54.864692 (+ 68us) server_negotiation.cc:407] Connection header received
0926 13:03:54.864779 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:54.864779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:54.864786 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:54.864824 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:54.865187 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.865854 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.866535 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.866704 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.866756 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:54.867582 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:54.867582 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:54.867582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:54.867587 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:54.868624 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.868965 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.868966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.868966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.869010 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:54.869150 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:54.869151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:54.869151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:54.869223 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:54.869263 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:54.869338 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 13:03:54.869389 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:54.869454 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:54.864478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.1:51733)
0926 13:03:54.864636 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:54.864647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:54.864705 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:54.864843 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:54.864844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:54.865066 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:54.865120 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:54.865121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.865947 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.865947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:54.866464 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:54.866466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:54.866768 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:54.866769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:54.866822 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:54.867248 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:54.867253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:54.867515 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:54.868691 (+ 1176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:54.868691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:54.868692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:54.868845 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:54.869064 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:54.869065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:54.869065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:54.869114 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:54.869255 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:54.869256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:54.869283 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 13:03:54.869326 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 13:03:54.869394 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260926 13:03:54.870416 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3576
I20260926 13:03:54.870515 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:03:54.870694 2392 external_mini_cluster.cc:1428] Setting key 0571a17ffbe39654d54faed67b547977
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:03:54.895126 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33523
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:54.973170 3716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:54.973393 3716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:54.973440 3716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:54.974545 3716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:54.974627 3716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:54.974661 3716 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:54.974705 3716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:03:54.976081 3716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:54.976348 3716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:54.976574 3716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:54.981276 3721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:54.981262 3722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:54.981307 3716 server_base.cc:1034] running on GCE node
W20260926 13:03:54.981262 3724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:54.981676 3716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:54.981873 3716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:54.983004 3716 hybrid_clock.cc:648] HybridClock initialized: now 1790427834982994 us; error 27 us; skew 500 ppm
Sep 26 13:03:54 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:54.985435 3716 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:03:54.985904 3716 webserver.cc:460] Webserver started at http://127.2.86.2:34453/ using document root <none> and password file <none>
I20260926 13:03:54.986128 3716 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:54.986191 3716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:54.986331 3716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:54.987295 3716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "98a592b542d8452d85af9389fed822f9"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "1489f1e5feae37dc965446cb9ed4d60f"
server_key_iv: "3b1576ba362b9ae612f7d42ab3c9af06"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.987673 3716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "98a592b542d8452d85af9389fed822f9"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "1489f1e5feae37dc965446cb9ed4d60f"
server_key_iv: "3b1576ba362b9ae612f7d42ab3c9af06"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.988962 3716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:03:54.989652 3731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:54.989830 3716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:54.989933 3716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "98a592b542d8452d85af9389fed822f9"
format_stamp: "Formatted at 2026-09-26 13:03:54 on dist-test-slave-j4lq"
server_key: "1489f1e5feae37dc965446cb9ed4d60f"
server_key_iv: "3b1576ba362b9ae612f7d42ab3c9af06"
server_key_version: "encryptionkey@0"
I20260926 13:03:54.990001 3716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:54.997581 3716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:54.999377 3716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:54.999531 3716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:54.999987 3716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:55.000053 3716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.000098 3716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:55.000126 3716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:55.006965 3716 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:46479
I20260926 13:03:55.007052 3846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:46479 every 8 connection(s)
I20260926 13:03:55.007356 3716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:03:55.009399 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3716
I20260926 13:03:55.009497 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:03:55.009688 2392 external_mini_cluster.cc:1428] Setting key 3ea3dbcfd4841df6bc7e6ce1b4fefc25
I20260926 13:03:55.010140 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.000512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:40343 (local address 127.2.86.62:33523)
0926 13:03:55.000581 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.000583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.000859 (+ 276us) server_negotiation.cc:407] Connection header received
0926 13:03:55.001385 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.001386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.001399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.001452 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.002232 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.003087 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.004149 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.004505 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.004573 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.007268 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.007269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.007269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.007275 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.008608 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.009164 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.009165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.009165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.009233 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.009489 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.009490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.009491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.009645 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.009717 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.010002 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.010081 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:03:55.010317 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.000478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.2:40343)
0926 13:03:55.000730 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.000757 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.001273 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.001518 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.001522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.001858 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:55.002124 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.002129 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.003223 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.003224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.004042 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.004044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.004629 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.004631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.004834 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.005401 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:55.005408 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:55.007170 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:55.008788 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.008794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.008795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.009098 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.009299 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.009300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.009300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.009408 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.009712 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:55.009716 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:55.009880 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.009960 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.010117 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":110,"threads_started":1}
I20260926 13:03:55.011994 3847 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33523
I20260926 13:03:55.012087 3847 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:55.012253 3847 heartbeater.cc:511] Master 127.2.86.62:33523 requested a full tablet report, sending...
I20260926 13:03:55.012571 3503 ts_manager.cc:194] Registered new tserver with Master: 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479)
I20260926 13:03:55.013019 3503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:40343
I20260926 13:03:55.017627 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.012007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:48047 (local address 127.2.86.62:33523)
0926 13:03:55.012052 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.012054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.012062 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:55.012085 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.012086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.012095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.012150 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.012489 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.013223 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.013972 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.014174 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.014244 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.015145 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.015146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.015147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.015153 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.016487 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.016858 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.016859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.016860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.016932 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.017120 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.017121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.017121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.017259 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.017337 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.017471 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.017543 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:55.017629 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.011840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.2:48047)
0926 13:03:55.011980 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.011992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.012039 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.012163 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.012164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.012361 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:55.012426 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.012428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.013318 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.013319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.013862 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.013864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.014255 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.014256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.014319 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.014758 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:55.014763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:55.015040 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:55.016578 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.016579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.016580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.016750 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.016992 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.016993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.016994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.017048 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.017329 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:55.017330 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:55.017373 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.017441 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.017536 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:03:55.035487 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33523
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:55.113722 3856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:55.113896 3856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:55.113919 3856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:55.114980 3856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:55.115012 3856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:55.115031 3856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:55.115049 3856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:03:55.116345 3856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:33523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:55.116506 3856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:55.116678 3856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:55.121199 3861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:55.121277 3862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:55.121305 3856 server_base.cc:1034] running on GCE node
W20260926 13:03:55.121196 3864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:55.121572 3856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:55.121819 3856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:55.122990 3856 hybrid_clock.cc:648] HybridClock initialized: now 1790427835122985 us; error 36 us; skew 500 ppm
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427835, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:55.125471 3856 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:03:55.125918 3856 webserver.cc:460] Webserver started at http://127.2.86.3:38985/ using document root <none> and password file <none>
I20260926 13:03:55.126099 3856 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:55.126138 3856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:55.126221 3856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:55.127136 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "405559cef2e24cdd81e80ba6545c6a54"
format_stamp: "Formatted at 2026-09-26 13:03:55 on dist-test-slave-j4lq"
server_key: "25010187cb189651485cc2c90c43e6e3"
server_key_iv: "fc87269d3521ab53297bc5731b7fd3b1"
server_key_version: "encryptionkey@0"
I20260926 13:03:55.127421 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "405559cef2e24cdd81e80ba6545c6a54"
format_stamp: "Formatted at 2026-09-26 13:03:55 on dist-test-slave-j4lq"
server_key: "25010187cb189651485cc2c90c43e6e3"
server_key_iv: "fc87269d3521ab53297bc5731b7fd3b1"
server_key_version: "encryptionkey@0"
I20260926 13:03:55.128537 3856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:03:55.129194 3871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.129334 3856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.129431 3856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "405559cef2e24cdd81e80ba6545c6a54"
format_stamp: "Formatted at 2026-09-26 13:03:55 on dist-test-slave-j4lq"
server_key: "25010187cb189651485cc2c90c43e6e3"
server_key_iv: "fc87269d3521ab53297bc5731b7fd3b1"
server_key_version: "encryptionkey@0"
I20260926 13:03:55.129479 3856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:55.138542 3856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:55.140770 3856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:55.140945 3856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:55.141363 3856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:55.141403 3856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.141425 3856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:55.141459 3856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.148285 3856 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:32837
I20260926 13:03:55.148438 3986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:32837 every 8 connection(s)
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427835, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:55.148697 3856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:03:55.149631 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 3856
I20260926 13:03:55.149737 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:03:55.149931 2392 external_mini_cluster.cc:1428] Setting key 0f2b2bade132bc7b6276e8e32669ccc9
I20260926 13:03:55.152004 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.141828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:58957 (local address 127.2.86.62:33523)
0926 13:03:55.141874 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.141876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.142070 (+ 194us) server_negotiation.cc:407] Connection header received
0926 13:03:55.142562 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.142564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.142576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.142640 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.143544 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.144359 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.146044 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.146380 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.146455 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.149199 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.149201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.149201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.149209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.150940 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.151287 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.151288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.151288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.151348 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.151548 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.151549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.151549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.151645 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.151693 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.151892 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.151946 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:03:55.152256 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.141818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.3:58957)
0926 13:03:55.141967 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.142002 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.142453 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.142752 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.142757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.143131 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:55.143439 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.143447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.144460 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.144462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.145938 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.145942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.146473 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.146474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.146662 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.147240 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:55.147251 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:55.149106 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:55.151011 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.151016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.151018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.151224 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.151404 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.151404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.151405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.151496 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.151697 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:55.151701 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:55.151805 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.151881 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.151997 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260926 13:03:55.152699 3987 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33523
I20260926 13:03:55.152778 3987 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:55.152993 3987 heartbeater.cc:511] Master 127.2.86.62:33523 requested a full tablet report, sending...
I20260926 13:03:55.153352 3504 ts_manager.cc:194] Registered new tserver with Master: 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837)
I20260926 13:03:55.153860 3504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:58957
I20260926 13:03:55.158797 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.153925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:49859 (local address 127.2.86.62:33523)
0926 13:03:55.153966 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.153968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.154099 (+ 131us) server_negotiation.cc:407] Connection header received
0926 13:03:55.154515 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.154515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.154525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.154578 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.154861 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.155292 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.155917 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.156081 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.156127 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.156853 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.156854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.156854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.156859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.158018 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.158297 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.158298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.158298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.158343 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.158481 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.158482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.158482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.158558 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.158600 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.158695 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.158745 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:55.158813 3993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.153829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.2.86.3:49859)
0926 13:03:55.154037 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.154048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.154101 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.154593 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.154594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.154741 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:55.154786 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.154787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.155368 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.155368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.155848 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.155849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.156139 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.156139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.156189 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.156512 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:55.156516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:55.156782 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:55.158084 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.158085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.158085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.158236 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.158396 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.158396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.158397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.158444 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.158602 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:55.158603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:55.158634 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.158682 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.158748 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260926 13:03:55.161510 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:55.191834 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.184287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.2.86.62:33523)
0926 13:03:55.184330 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.184332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.184359 (+ 27us) server_negotiation.cc:407] Connection header received
0926 13:03:55.184399 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.184400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.184409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.184490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.185401 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.186032 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.187113 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.187378 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.187457 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.189754 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.189755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.189755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.189762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.190939 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.191276 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.191277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.191277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.191325 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.191478 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.191479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.191479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.191571 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.191642 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.191722 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.191774 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:03:55.193557 3504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:03:55.195138 3504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:55.201656 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46479 (local address 127.0.0.1:36310)
0926 13:03:55.197647 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.197689 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.197789 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.198094 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.198097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.198099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.198163 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.198165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199424 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.199425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.200387 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.200389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201341 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.201426 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201438 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.201524 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.201577 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":9,"thread_start_us":65,"threads_started":1}
I20260926 13:03:55.201843 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.2.86.3:32837)
0926 13:03:55.197943 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.197948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.197956 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:55.197984 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.197986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.197998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.198085 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.198406 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199191 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.200642 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201279 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201374 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201443 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.201693 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.201738 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260926 13:03:55.201974 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.2.86.1:40515)
0926 13:03:55.197919 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.197924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.198231 (+ 307us) server_negotiation.cc:407] Connection header received
0926 13:03:55.198304 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.198307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.198322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.198396 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.198494 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199492 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.200730 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201440 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201593 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.201688 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.201722 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260926 13:03:55.202045 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.2.86.2:46479)
0926 13:03:55.197637 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.197645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.197756 (+ 111us) server_negotiation.cc:407] Connection header received
0926 13:03:55.197897 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.197900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.197917 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.197981 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.198245 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199320 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.200468 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201246 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201322 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201379 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.201816 (+ 437us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.201859 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260926 13:03:55.201781 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40515 (local address 127.0.0.1:42720)
0926 13:03:55.198168 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.198177 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.198230 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.198393 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.198394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.198397 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.198436 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.198437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199568 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.199569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.200654 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.200656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201552 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.201614 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201618 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.201674 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.201714 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":84,"threads_started":1}
I20260926 13:03:55.202165 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.197377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:32837 (local address 127.0.0.1:50022)
0926 13:03:55.197711 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.197722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.197785 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.198102 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.198103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.198105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.198179 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.198181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.199269 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.199270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.200544 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.200547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.201474 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.201475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.201550 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.201553 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.201597 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.201628 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":32,"thread_start_us":73,"threads_started":1}
I20260926 13:03:55.202941 3641 tablet_service.cc:1467] Processing CreateTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 (DEFAULT_TABLE table=test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.202980 3781 tablet_service.cc:1467] Processing CreateTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 (DEFAULT_TABLE table=test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.202982 3921 tablet_service.cc:1467] Processing CreateTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 (DEFAULT_TABLE table=test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.203218 3641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ab7c5ca4242d1b0e6af83c59cd426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.203293 3921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ab7c5ca4242d1b0e6af83c59cd426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.203292 3781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ab7c5ca4242d1b0e6af83c59cd426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.206404 4010 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Bootstrap starting.
I20260926 13:03:55.206640 4011 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Bootstrap starting.
I20260926 13:03:55.206777 4012 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Bootstrap starting.
I20260926 13:03:55.207155 4010 tablet_bootstrap.cc:654] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.207363 4012 tablet_bootstrap.cc:654] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.207455 4010 log.cc:824] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:55.207513 4011 tablet_bootstrap.cc:654] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.207649 4012 log.cc:824] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:55.207809 4011 log.cc:824] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:55.208256 4010 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: No bootstrap required, opened a new log
I20260926 13:03:55.208341 4010 ts_tablet_manager.cc:1397] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:55.208380 4012 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: No bootstrap required, opened a new log
I20260926 13:03:55.208444 4011 tablet_bootstrap.cc:492] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: No bootstrap required, opened a new log
I20260926 13:03:55.208460 4012 ts_tablet_manager.cc:1397] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:55.208499 4011 ts_tablet_manager.cc:1397] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:55.209827 4010 raft_consensus.cc:348] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.209996 4010 raft_consensus.cc:374] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.210038 4010 raft_consensus.cc:729] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfe6bbe1c5414c4cbedf3974a2af6002, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.210142 4010 consensus_queue.cc:260] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.210420 4010 ts_tablet_manager.cc:1428] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:03:55.210378 4011 raft_consensus.cc:348] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.210378 4012 raft_consensus.cc:348] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.210532 4012 raft_consensus.cc:374] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.210532 4011 raft_consensus.cc:374] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.210563 3707 heartbeater.cc:503] Master 127.2.86.62:33523 was elected leader, sending a full tablet report...
I20260926 13:03:55.210574 4012 raft_consensus.cc:729] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a592b542d8452d85af9389fed822f9, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.210606 4011 raft_consensus.cc:729] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405559cef2e24cdd81e80ba6545c6a54, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.210696 4012 consensus_queue.cc:260] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.210738 4011 consensus_queue.cc:260] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.210983 4011 ts_tablet_manager.cc:1428] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:03:55.210983 4012 ts_tablet_manager.cc:1428] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:03:55.211095 3987 heartbeater.cc:503] Master 127.2.86.62:33523 was elected leader, sending a full tablet report...
I20260926 13:03:55.211112 3847 heartbeater.cc:503] Master 127.2.86.62:33523 was elected leader, sending a full tablet report...
W20260926 13:03:55.258025 3848 tablet.cc:2369] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:03:55.306265 4017 raft_consensus.cc:482] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:55.306412 4017 raft_consensus.cc:504] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.306730 4017 leader_election.cc:288] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837), bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515)
I20260926 13:03:55.309499 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.306938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40515 (local address 127.2.86.2:43617)
0926 13:03:55.306998 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.307011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.307061 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.307242 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.307243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.307245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.307315 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.307317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.307937 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.307938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.308662 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.308664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.309243 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.309244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.309320 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.309323 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.309390 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.309431 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:03:55.309499 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.307083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:43617 (local address 127.2.86.1:40515)
0926 13:03:55.307128 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.307130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.307137 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:03:55.307168 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.307168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.307176 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.307238 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.307390 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.307857 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.308756 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.309155 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.309235 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.309244 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.309396 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.309431 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:03:55.309756 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.307033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:43621 (local address 127.2.86.3:32837)
0926 13:03:55.307090 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.307092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.307319 (+ 227us) server_negotiation.cc:407] Connection header received
0926 13:03:55.307391 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.307393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.307403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.307479 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.307633 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.308088 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.309019 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.309360 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.309454 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.309462 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.309643 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.309693 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:03:55.309906 3661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ab7c5ca4242d1b0e6af83c59cd426" candidate_uuid: "98a592b542d8452d85af9389fed822f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" is_pre_election: true
I20260926 13:03:55.309999 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.306948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:32837 (local address 127.2.86.2:43621)
0926 13:03:55.307234 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.307244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.307326 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.307483 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.307484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.307485 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.307562 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.307563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.308173 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.308174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.308888 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.308890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.309455 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.309456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.309527 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.309536 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.309873 (+ 337us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.309922 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":96,"thread_start_us":136,"threads_started":1}
I20260926 13:03:55.310075 3661 raft_consensus.cc:2454] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98a592b542d8452d85af9389fed822f9 in term 0.
I20260926 13:03:55.310309 3734 leader_election.cc:302] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98a592b542d8452d85af9389fed822f9, bfe6bbe1c5414c4cbedf3974a2af6002; no voters:
I20260926 13:03:55.310305 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ab7c5ca4242d1b0e6af83c59cd426" candidate_uuid: "98a592b542d8452d85af9389fed822f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405559cef2e24cdd81e80ba6545c6a54" is_pre_election: true
I20260926 13:03:55.310439 3941 raft_consensus.cc:2454] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98a592b542d8452d85af9389fed822f9 in term 0.
I20260926 13:03:55.310431 4017 raft_consensus.cc:2781] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:55.310482 4017 raft_consensus.cc:482] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:55.310508 4017 raft_consensus.cc:3037] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.311667 4017 raft_consensus.cc:504] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.311820 4017 leader_election.cc:288] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [CANDIDATE]: Term 1 election: Requested vote from peers 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837), bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515)
I20260926 13:03:55.311956 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ab7c5ca4242d1b0e6af83c59cd426" candidate_uuid: "98a592b542d8452d85af9389fed822f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405559cef2e24cdd81e80ba6545c6a54"
I20260926 13:03:55.311951 3661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ab7c5ca4242d1b0e6af83c59cd426" candidate_uuid: "98a592b542d8452d85af9389fed822f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002"
I20260926 13:03:55.312033 3941 raft_consensus.cc:3037] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.312057 3661 raft_consensus.cc:3037] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.312749 3661 raft_consensus.cc:2454] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98a592b542d8452d85af9389fed822f9 in term 1.
I20260926 13:03:55.312791 3941 raft_consensus.cc:2454] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98a592b542d8452d85af9389fed822f9 in term 1.
I20260926 13:03:55.312994 3734 leader_election.cc:302] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98a592b542d8452d85af9389fed822f9, bfe6bbe1c5414c4cbedf3974a2af6002; no voters:
I20260926 13:03:55.313099 4017 raft_consensus.cc:2781] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:55.313251 4017 raft_consensus.cc:686] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 1 LEADER]: Becoming Leader. State: Replica: 98a592b542d8452d85af9389fed822f9, State: Running, Role: LEADER
I20260926 13:03:55.313470 4017 consensus_queue.cc:237] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } }
I20260926 13:03:55.314248 3503 catalog_manager.cc:5495] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98a592b542d8452d85af9389fed822f9 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "98a592b542d8452d85af9389fed822f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } health_report { overall_health: UNKNOWN } } }
I20260926 13:03:55.344796 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.342646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.2.86.2:46479)
0926 13:03:55.342705 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.342707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.342715 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:55.342744 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.342744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.342761 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.342816 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:55.342957 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.343388 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.344079 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.344246 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.344292 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.344450 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:55.344533 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:55.344585 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.344637 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.344653 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:03:55.346603 3661 raft_consensus.cc:1261] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Refusing update from remote peer 98a592b542d8452d85af9389fed822f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:55.346603 3941 raft_consensus.cc:1261] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Refusing update from remote peer 98a592b542d8452d85af9389fed822f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:55.346987 4020 consensus_queue.cc:1035] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:55.347149 4017 consensus_queue.cc:1035] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:55.350335 4025 mvcc.cc:205] Tried to move back new op lower bound from 7333592413576863744 to 7333592413444562944. Current Snapshot: MvccSnapshot[applied={T|T < 7333592413576863744}]
I20260926 13:03:55.352782 3503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:55.352923 3503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:55.353713 3503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Sending DeleteTablet for 3 replicas of tablet 794ab7c5ca4242d1b0e6af83c59cd426
I20260926 13:03:55.354125 3781 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:55 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36310
I20260926 13:03:55.354102 3921 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:55 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:50022
I20260926 13:03:55.354216 3641 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ab7c5ca4242d1b0e6af83c59cd426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:55 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42720
I20260926 13:03:55.354288 3503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:03:55.354420 3503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:55.354429 4033 tablet_replica.cc:330] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: stopping tablet replica
I20260926 13:03:55.354527 4033 raft_consensus.cc:2229] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:55.354614 4033 raft_consensus.cc:2258] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:55.354774 4032 tablet_replica.cc:330] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: stopping tablet replica
I20260926 13:03:55.354857 4032 raft_consensus.cc:2229] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:55.354961 4032 raft_consensus.cc:2258] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:55.355470 4032 ts_tablet_manager.cc:1905] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:55.355772 3641 tablet_service.cc:1467] Processing CreateTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 (DEFAULT_TABLE table=test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.355793 3781 tablet_service.cc:1467] Processing CreateTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 (DEFAULT_TABLE table=test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.355896 3641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7dbdf8ddbb348f6b507c3793a836c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.355921 3781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7dbdf8ddbb348f6b507c3793a836c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.356220 3921 tablet_service.cc:1467] Processing CreateTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 (DEFAULT_TABLE table=test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:55.356307 3921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7dbdf8ddbb348f6b507c3793a836c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.357326 4012 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Bootstrap starting.
I20260926 13:03:55.357324 4010 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Bootstrap starting.
I20260926 13:03:55.357659 4011 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Bootstrap starting.
I20260926 13:03:55.357733 4012 tablet_bootstrap.cc:654] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.357999 4010 tablet_bootstrap.cc:654] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.358074 4011 tablet_bootstrap.cc:654] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.358451 4012 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: No bootstrap required, opened a new log
I20260926 13:03:55.358500 4012 ts_tablet_manager.cc:1397] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:55.358641 4033 ts_tablet_manager.cc:1905] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:55.358629 4012 raft_consensus.cc:348] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.358696 4012 raft_consensus.cc:374] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.358718 4012 raft_consensus.cc:729] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a592b542d8452d85af9389fed822f9, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.358747 4010 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: No bootstrap required, opened a new log
I20260926 13:03:55.358758 4012 consensus_queue.cc:260] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.358783 4010 ts_tablet_manager.cc:1397] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:55.358839 4012 ts_tablet_manager.cc:1428] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.358933 4010 raft_consensus.cc:348] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.359012 4010 raft_consensus.cc:374] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.359033 4010 raft_consensus.cc:729] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfe6bbe1c5414c4cbedf3974a2af6002, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.359069 4010 consensus_queue.cc:260] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.359141 4010 ts_tablet_manager.cc:1428] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.359889 4011 tablet_bootstrap.cc:492] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: No bootstrap required, opened a new log
I20260926 13:03:55.359947 4011 ts_tablet_manager.cc:1397] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:55.360095 4011 raft_consensus.cc:348] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.360162 4033 ts_tablet_manager.cc:1918] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:03:55.360183 4011 raft_consensus.cc:374] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.360208 4011 raft_consensus.cc:729] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405559cef2e24cdd81e80ba6545c6a54, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.360219 4033 log.cc:1196] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/794ab7c5ca4242d1b0e6af83c59cd426
I20260926 13:03:55.360255 4011 consensus_queue.cc:260] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.360363 4011 ts_tablet_manager.cc:1428] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:55.360486 4033 ts_tablet_manager.cc:1939] T 794ab7c5ca4242d1b0e6af83c59cd426 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting consensus metadata
I20260926 13:03:55.360901 3490 catalog_manager.cc:4841] TS bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515): tablet 794ab7c5ca4242d1b0e6af83c59cd426 (table test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]) successfully deleted
I20260926 13:03:55.361028 4034 tablet_replica.cc:330] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: stopping tablet replica
W20260926 13:03:55.361106 3708 tablet.cc:2369] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:03:55.361205 4034 raft_consensus.cc:2229] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:03:55.361364 4034 raft_consensus.cc:2258] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:55.362022 4034 ts_tablet_manager.cc:1905] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:55.362046 4032 ts_tablet_manager.cc:1918] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:03:55.362108 4032 log.cc:1196] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/794ab7c5ca4242d1b0e6af83c59cd426
I20260926 13:03:55.362349 4032 ts_tablet_manager.cc:1939] T 794ab7c5ca4242d1b0e6af83c59cd426 P 405559cef2e24cdd81e80ba6545c6a54: Deleting consensus metadata
I20260926 13:03:55.362709 3489 catalog_manager.cc:4841] TS 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837): tablet 794ab7c5ca4242d1b0e6af83c59cd426 (table test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]) successfully deleted
I20260926 13:03:55.364122 4034 ts_tablet_manager.cc:1918] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:03:55.364194 4034 log.cc:1196] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/794ab7c5ca4242d1b0e6af83c59cd426
I20260926 13:03:55.364439 4034 ts_tablet_manager.cc:1939] T 794ab7c5ca4242d1b0e6af83c59cd426 P 98a592b542d8452d85af9389fed822f9: Deleting consensus metadata
I20260926 13:03:55.365150 3490 catalog_manager.cc:4841] TS 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479): tablet 794ab7c5ca4242d1b0e6af83c59cd426 (table test-table [id=bb1b28c2c93e43dc9882deb4d2030d39]) successfully deleted
W20260926 13:03:55.400007 3988 tablet.cc:2369] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:03:55.417726 4018 raft_consensus.cc:482] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:55.417825 4018 raft_consensus.cc:504] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.418146 4018 leader_election.cc:288] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515), 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479)
I20260926 13:03:55.421664 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.418342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40515 (local address 127.2.86.3:32909)
0926 13:03:55.418423 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.418459 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.418505 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.418721 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.418722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.418723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.418822 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.418824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.419694 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.419695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.420722 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.420725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.421356 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.421357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.421470 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.421474 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.421531 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.421577 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 13:03:55.421677 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.418416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:32909 (local address 127.2.86.1:40515)
0926 13:03:55.418481 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.418483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.418494 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:03:55.418566 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.418567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.418577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.418643 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.418892 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.419602 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.420859 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.421264 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.421352 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.421362 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.421560 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.421610 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:03:55.421749 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.418659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:39963 (local address 127.2.86.2:46479)
0926 13:03:55.418719 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.418721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.418729 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:55.418753 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.418754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.418762 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.418814 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.418950 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.419405 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.420821 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.421380 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.421463 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.421473 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.421639 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.421678 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:03:55.422029 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.418527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46479 (local address 127.2.86.3:39963)
0926 13:03:55.418646 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.418655 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.418695 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.418828 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.418829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.418831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.418892 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.418893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.419492 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.419493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.420722 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.420725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.421483 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.421484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.421553 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.421556 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.421870 (+ 314us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.421919 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260926 13:03:55.422134 3661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7dbdf8ddbb348f6b507c3793a836c72" candidate_uuid: "405559cef2e24cdd81e80ba6545c6a54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" is_pre_election: true
I20260926 13:03:55.422238 3661 raft_consensus.cc:2454] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 405559cef2e24cdd81e80ba6545c6a54 in term 0.
I20260926 13:03:55.422281 3801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7dbdf8ddbb348f6b507c3793a836c72" candidate_uuid: "405559cef2e24cdd81e80ba6545c6a54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a592b542d8452d85af9389fed822f9" is_pre_election: true
I20260926 13:03:55.422415 3801 raft_consensus.cc:2454] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 405559cef2e24cdd81e80ba6545c6a54 in term 0.
I20260926 13:03:55.422411 3874 leader_election.cc:302] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405559cef2e24cdd81e80ba6545c6a54, bfe6bbe1c5414c4cbedf3974a2af6002; no voters:
I20260926 13:03:55.422524 4018 raft_consensus.cc:2781] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:55.422596 4018 raft_consensus.cc:482] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:55.422619 4018 raft_consensus.cc:3037] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.423519 4018 raft_consensus.cc:504] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.423662 4018 leader_election.cc:288] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [CANDIDATE]: Term 1 election: Requested vote from peers bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515), 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479)
I20260926 13:03:55.423817 3661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7dbdf8ddbb348f6b507c3793a836c72" candidate_uuid: "405559cef2e24cdd81e80ba6545c6a54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002"
I20260926 13:03:55.423897 3661 raft_consensus.cc:3037] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.423879 3801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7dbdf8ddbb348f6b507c3793a836c72" candidate_uuid: "405559cef2e24cdd81e80ba6545c6a54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a592b542d8452d85af9389fed822f9"
I20260926 13:03:55.423985 3801 raft_consensus.cc:3037] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.424634 3801 raft_consensus.cc:2454] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 405559cef2e24cdd81e80ba6545c6a54 in term 1.
I20260926 13:03:55.424643 3661 raft_consensus.cc:2454] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 405559cef2e24cdd81e80ba6545c6a54 in term 1.
I20260926 13:03:55.424803 3874 leader_election.cc:302] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405559cef2e24cdd81e80ba6545c6a54, 98a592b542d8452d85af9389fed822f9; no voters:
I20260926 13:03:55.424923 4018 raft_consensus.cc:2781] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:55.425078 4018 raft_consensus.cc:686] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 LEADER]: Becoming Leader. State: Replica: 405559cef2e24cdd81e80ba6545c6a54, State: Running, Role: LEADER
I20260926 13:03:55.425148 4018 consensus_queue.cc:237] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } }
I20260926 13:03:55.425956 3504 catalog_manager.cc:5495] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "405559cef2e24cdd81e80ba6545c6a54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } health_report { overall_health: HEALTHY } } }
Sep 26 13:03:55 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427834, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:55.461097 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.453711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.2.86.62:33523)
0926 13:03:55.453765 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.453768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.453933 (+ 165us) server_negotiation.cc:407] Connection header received
0926 13:03:55.453962 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.453964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.453976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.454037 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:55.454474 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.455240 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.455934 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.456211 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.456291 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.458552 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:55.458553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:55.458554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:55.458560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:55.459982 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.460334 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.460334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.460335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.460392 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:55.460565 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:55.460566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:55.460567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:55.460685 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:55.460745 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.460891 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.460977 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 13:03:55.461176 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.453636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33523 (local address 127.0.0.1:33824)
0926 13:03:55.453797 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.453812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.453880 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.454048 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.454049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.454326 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:55.454381 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.454383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.455340 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.455341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.455846 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.455847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.456293 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.456293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.456340 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.456941 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:55.456946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:55.458459 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:55.460068 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.460074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.460076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.460261 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.460455 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:55.460456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:55.460456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:55.460509 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:55.460742 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:55.460747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:55.460785 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.460842 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.461045 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260926 13:03:55.463162 3504 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33824:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:03:55.463408 3504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:55.464818 3641 tablet_service.cc:1467] Processing CreateTablet for tablet cad8a59c7ced4e11b487a5837e6a91bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=842d1bf04ada433c87e96e0571024d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:55.464844 3921 tablet_service.cc:1467] Processing CreateTablet for tablet cad8a59c7ced4e11b487a5837e6a91bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=842d1bf04ada433c87e96e0571024d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:55.464892 3781 tablet_service.cc:1467] Processing CreateTablet for tablet cad8a59c7ced4e11b487a5837e6a91bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=842d1bf04ada433c87e96e0571024d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:55.465004 3921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad8a59c7ced4e11b487a5837e6a91bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.465004 3641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad8a59c7ced4e11b487a5837e6a91bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.465029 3781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cad8a59c7ced4e11b487a5837e6a91bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:55.466459 4010 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: Bootstrap starting.
I20260926 13:03:55.466459 4011 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: Bootstrap starting.
I20260926 13:03:55.466850 4010 tablet_bootstrap.cc:654] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.466861 4011 tablet_bootstrap.cc:654] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.466878 4012 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: Bootstrap starting.
I20260926 13:03:55.467262 4012 tablet_bootstrap.cc:654] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:55.467638 4011 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: No bootstrap required, opened a new log
I20260926 13:03:55.467691 4011 ts_tablet_manager.cc:1397] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:55.467729 4010 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: No bootstrap required, opened a new log
I20260926 13:03:55.467773 4010 ts_tablet_manager.cc:1397] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:55.467852 4011 raft_consensus.cc:348] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.467921 4011 raft_consensus.cc:374] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.467941 4011 raft_consensus.cc:729] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405559cef2e24cdd81e80ba6545c6a54, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.467900 4010 raft_consensus.cc:348] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.467962 4010 raft_consensus.cc:374] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.467983 4010 raft_consensus.cc:729] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfe6bbe1c5414c4cbedf3974a2af6002, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.467986 4011 consensus_queue.cc:260] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.468024 4010 consensus_queue.cc:260] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.468062 4012 tablet_bootstrap.cc:492] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: No bootstrap required, opened a new log
I20260926 13:03:55.468073 4011 ts_tablet_manager.cc:1428] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.468102 4012 ts_tablet_manager.cc:1397] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:55.468119 4010 ts_tablet_manager.cc:1428] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.468191 4012 raft_consensus.cc:348] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.468247 4012 raft_consensus.cc:374] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:55.468214 4018 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:55.468266 4012 raft_consensus.cc:729] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a592b542d8452d85af9389fed822f9, State: Initialized, Role: FOLLOWER
I20260926 13:03:55.468278 4018 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:55.468319 4012 consensus_queue.cc:260] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.468402 4016 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:55.468420 4017 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:55.468477 4016 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:55.468506 4017 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:55.468408 4012 ts_tablet_manager.cc:1428] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:55.835450 4018 consensus_queue.cc:1035] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:55.837188 4051 consensus_queue.cc:1035] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:55.898684 4016 raft_consensus.cc:482] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:55.898793 4016 raft_consensus.cc:504] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.899191 4016 leader_election.cc:288] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837), 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479)
I20260926 13:03:55.902192 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.899475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41631 (local address 127.2.86.2:46479)
0926 13:03:55.899521 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.899523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.899648 (+ 125us) server_negotiation.cc:407] Connection header received
0926 13:03:55.899729 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.899730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.899741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.899784 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.899973 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.900431 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901511 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.901855 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901935 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.901945 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.902087 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.902123 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:55.902209 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.899429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46479 (local address 127.2.86.1:41631)
0926 13:03:55.899567 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.899578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.899647 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.899805 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.899806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.899808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.899885 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.899886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.900530 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.900531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.901390 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.901393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.901934 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.902008 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.902012 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.902070 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.902109 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260926 13:03:55.902218 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.899499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:33165 (local address 127.2.86.3:32837)
0926 13:03:55.899594 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:55.899597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:55.899607 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:03:55.899704 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:55.899705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:55.899718 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:55.899815 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:55.899961 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.900440 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901482 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.901863 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901962 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.901973 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:55.902106 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 13:03:55.902152 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":45,"threads_started":1}
I20260926 13:03:55.902420 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:55.899425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:32837 (local address 127.2.86.1:33165)
0926 13:03:55.899550 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:55.899560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:55.899614 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:55.899833 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:55.899834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:55.899836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:55.899883 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.899885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.900551 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.900551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.901386 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:55.901389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:55.901967 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:55.901968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:55.902024 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:55.902027 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:55.902098 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 13:03:55.902138 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260926 13:03:55.902580 3801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad8a59c7ced4e11b487a5837e6a91bf" candidate_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a592b542d8452d85af9389fed822f9" is_pre_election: true
I20260926 13:03:55.902695 3801 raft_consensus.cc:2454] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfe6bbe1c5414c4cbedf3974a2af6002 in term 0.
I20260926 13:03:55.902661 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad8a59c7ced4e11b487a5837e6a91bf" candidate_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405559cef2e24cdd81e80ba6545c6a54" is_pre_election: true
I20260926 13:03:55.902765 3941 raft_consensus.cc:2454] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfe6bbe1c5414c4cbedf3974a2af6002 in term 0.
I20260926 13:03:55.902855 3594 leader_election.cc:302] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98a592b542d8452d85af9389fed822f9, bfe6bbe1c5414c4cbedf3974a2af6002; no voters:
I20260926 13:03:55.903012 4016 raft_consensus.cc:2781] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:55.903050 4016 raft_consensus.cc:482] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:55.903064 4016 raft_consensus.cc:3037] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.903735 4016 raft_consensus.cc:504] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.903878 4016 leader_election.cc:288] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [CANDIDATE]: Term 1 election: Requested vote from peers 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837), 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479)
I20260926 13:03:55.904017 3941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad8a59c7ced4e11b487a5837e6a91bf" candidate_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405559cef2e24cdd81e80ba6545c6a54"
I20260926 13:03:55.904093 3941 raft_consensus.cc:3037] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.904759 3941 raft_consensus.cc:2454] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfe6bbe1c5414c4cbedf3974a2af6002 in term 1.
I20260926 13:03:55.904908 3801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cad8a59c7ced4e11b487a5837e6a91bf" candidate_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a592b542d8452d85af9389fed822f9"
I20260926 13:03:55.904992 3801 raft_consensus.cc:3037] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:55.904973 3593 leader_election.cc:302] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405559cef2e24cdd81e80ba6545c6a54, bfe6bbe1c5414c4cbedf3974a2af6002; no voters:
I20260926 13:03:55.905069 4016 raft_consensus.cc:2781] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:55.905175 4016 raft_consensus.cc:686] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 LEADER]: Becoming Leader. State: Replica: bfe6bbe1c5414c4cbedf3974a2af6002, State: Running, Role: LEADER
I20260926 13:03:55.905256 4016 consensus_queue.cc:237] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } }
I20260926 13:03:55.905543 4016 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: TxnStatusTablet state changed. Reason: New leader bfe6bbe1c5414c4cbedf3974a2af6002. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:55.905614 4016 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:55.905800 3801 raft_consensus.cc:2454] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfe6bbe1c5414c4cbedf3974a2af6002 in term 1.
I20260926 13:03:55.905998 4058 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:03:55.906111 4058 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:03:55.906033 3503 catalog_manager.cc:5495] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260926 13:03:56.020402 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.017255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.2.86.1:40515)
0926 13:03:56.017371 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.017378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.017388 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:03:56.017443 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.017444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.017460 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.017525 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:56.017698 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.018222 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.019317 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.019516 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.019571 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.020099 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.020186 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.020245 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.020322 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.020339 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
I20260926 13:03:56.020452 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.017150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40515 (local address 127.0.0.1:42732)
0926 13:03:56.017294 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:56.017306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:56.017366 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:56.017544 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:56.017545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:56.017547 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:56.017598 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.017599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.018317 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.018318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.019229 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.019231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.019589 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.019590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.019648 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.019663 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.020257 (+ 594us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.020267 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 13:03:56.020321 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:03:56.020370 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260926 13:03:56.021502 3941 raft_consensus.cc:1261] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Refusing update from remote peer bfe6bbe1c5414c4cbedf3974a2af6002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:56.021502 3801 raft_consensus.cc:1261] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Refusing update from remote peer bfe6bbe1c5414c4cbedf3974a2af6002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:56.021747 4056 consensus_queue.cc:1035] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:56.021905 4016 consensus_queue.cc:1035] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:56.022297 4051 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: TxnStatusTablet state changed. Reason: New leader bfe6bbe1c5414c4cbedf3974a2af6002. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.022404 4051 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:56.022784 4017 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: TxnStatusTablet state changed. Reason: New leader bfe6bbe1c5414c4cbedf3974a2af6002. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.022871 4017 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:56.023196 4016 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.023276 4016 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:56.023340 4024 mvcc.cc:205] Tried to move back new op lower bound from 7333592416342171648 to 7333592415868493824. Current Snapshot: MvccSnapshot[applied={T|T < 7333592416342171648}]
I20260926 13:03:56.023532 4016 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.023607 4016 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P bfe6bbe1c5414c4cbedf3974a2af6002: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:56.023808 4051 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.023895 4051 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 405559cef2e24cdd81e80ba6545c6a54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:56.024308 4023 mvcc.cc:205] Tried to move back new op lower bound from 7333592416342171648 to 7333592415868493824. Current Snapshot: MvccSnapshot[applied={T|T < 7333592416342171648}]
I20260926 13:03:56.024442 4017 tablet_replica.cc:439] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfe6bbe1c5414c4cbedf3974a2af6002" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40515 } } peers { permanent_uuid: "405559cef2e24cdd81e80ba6545c6a54" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 32837 } } peers { permanent_uuid: "98a592b542d8452d85af9389fed822f9" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46479 } } }
I20260926 13:03:56.024551 4017 tablet_replica.cc:442] T cad8a59c7ced4e11b487a5837e6a91bf P 98a592b542d8452d85af9389fed822f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:56.027709 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.025402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.2.86.3:32837)
0926 13:03:56.025440 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.025443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.025477 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:03:56.025552 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.025553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.025571 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.025665 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:56.025786 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.026245 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.026958 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.027157 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.027232 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.027413 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.027498 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.027573 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.027621 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.027639 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:03:56.032688 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.030253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:51143 (local address 127.2.86.1:40515)
0926 13:03:56.030313 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.030314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.030452 (+ 138us) server_negotiation.cc:407] Connection header received
0926 13:03:56.030535 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.030536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.030544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.030582 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:56.030678 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.031088 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.032012 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.032197 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.032283 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.032452 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.032499 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.032556 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.032615 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.032632 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:03:56.032735 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.030189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40515 (local address 127.2.86.3:51143)
0926 13:03:56.030394 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:56.030404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:56.030459 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:56.030593 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:56.030594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:56.030595 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:56.030632 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.030633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.031173 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.031174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.031932 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.031934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.032285 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.032285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.032345 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.032358 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.032558 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.032560 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:03:56.032610 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:03:56.032651 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260926 13:03:56.040910 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.038666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:32837 (local address 127.2.86.1:49933)
0926 13:03:56.038797 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:56.038806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:56.038864 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:56.038947 (+ 83us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:56.038948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:56.038949 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:56.038979 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.038979 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.039492 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.039493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.040058 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:56.040060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.040473 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.040473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:56.040538 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.040550 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.040737 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.040739 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:03:56.040776 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 13:03:56.040818 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260926 13:03:56.040891 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.038697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:49933 (local address 127.2.86.3:32837)
0926 13:03:56.038737 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.038739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.038852 (+ 113us) server_negotiation.cc:407] Connection header received
0926 13:03:56.038900 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.038901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.038908 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.038946 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:56.039014 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.039417 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.040171 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.040397 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.040473 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.040644 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:56.040673 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:56.040722 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.040788 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.040805 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:03:56.050019 3502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:56.050124 3502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:56.050838 3502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4c26289b3544889ab3bacc8eff4281a: Sending DeleteTablet for 3 replicas of tablet a7dbdf8ddbb348f6b507c3793a836c72
I20260926 13:03:56.051086 3921 tablet_service.cc:1514] Processing DeleteTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:56 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:50022
I20260926 13:03:56.051261 3781 tablet_service.cc:1514] Processing DeleteTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:56 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36310
I20260926 13:03:56.051451 3641 tablet_service.cc:1514] Processing DeleteTablet for tablet a7dbdf8ddbb348f6b507c3793a836c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:56 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42720
I20260926 13:03:56.051471 4078 tablet_replica.cc:330] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: stopping tablet replica
I20260926 13:03:56.051527 4078 raft_consensus.cc:2229] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:03:56.051594 4078 raft_consensus.cc:2258] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:56.051657 4081 tablet_replica.cc:330] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: stopping tablet replica
I20260926 13:03:56.051714 4081 raft_consensus.cc:2229] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:56.051765 4081 raft_consensus.cc:2258] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:56.051846 4078 ts_tablet_manager.cc:1905] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:56.052016 4081 ts_tablet_manager.cc:1905] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:56.053773 4081 ts_tablet_manager.cc:1918] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:56.053766 4078 ts_tablet_manager.cc:1918] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:56.053828 4078 log.cc:1196] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/a7dbdf8ddbb348f6b507c3793a836c72
I20260926 13:03:56.053828 4081 log.cc:1196] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/a7dbdf8ddbb348f6b507c3793a836c72
I20260926 13:03:56.054057 4078 ts_tablet_manager.cc:1939] T a7dbdf8ddbb348f6b507c3793a836c72 P 405559cef2e24cdd81e80ba6545c6a54: Deleting consensus metadata
I20260926 13:03:56.054057 4081 ts_tablet_manager.cc:1939] T a7dbdf8ddbb348f6b507c3793a836c72 P bfe6bbe1c5414c4cbedf3974a2af6002: Deleting consensus metadata
I20260926 13:03:56.054337 3490 catalog_manager.cc:4841] TS bfe6bbe1c5414c4cbedf3974a2af6002 (127.2.86.1:40515): tablet a7dbdf8ddbb348f6b507c3793a836c72 (table test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]) successfully deleted
I20260926 13:03:56.054387 3489 catalog_manager.cc:4841] TS 405559cef2e24cdd81e80ba6545c6a54 (127.2.86.3:32837): tablet a7dbdf8ddbb348f6b507c3793a836c72 (table test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]) successfully deleted
I20260926 13:03:56.056979 4079 tablet_replica.cc:330] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: stopping tablet replica
I20260926 13:03:56.057056 4079 raft_consensus.cc:2229] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:56.057101 4079 raft_consensus.cc:2258] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:56.057332 4079 ts_tablet_manager.cc:1905] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:56.058915 4079 ts_tablet_manager.cc:1918] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:56.058979 4079 log.cc:1196] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/a7dbdf8ddbb348f6b507c3793a836c72
I20260926 13:03:56.059190 4079 ts_tablet_manager.cc:1939] T a7dbdf8ddbb348f6b507c3793a836c72 P 98a592b542d8452d85af9389fed822f9: Deleting consensus metadata
I20260926 13:03:56.059635 3490 catalog_manager.cc:4841] TS 98a592b542d8452d85af9389fed822f9 (127.2.86.2:46479): tablet a7dbdf8ddbb348f6b507c3793a836c72 (table test-table [id=d3f0f15b9cca4a82aa753dfa616d6d87]) successfully deleted
Sep 26 13:03:56 dist-test-slave-j4lq krb5kdc[3459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427835, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:03:56.065951 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.057382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.2.86.1:40515)
0926 13:03:56.057423 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.057425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.057535 (+ 110us) server_negotiation.cc:407] Connection header received
0926 13:03:56.057598 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.057598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.057610 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.057665 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:56.058195 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.058652 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.059526 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.059842 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.059922 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.063040 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:56.063047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:56.063047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:56.063055 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:56.064895 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:56.065200 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:56.065203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:56.065204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:56.065268 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:56.065451 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:56.065452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:56.065452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:56.065608 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:56.065740 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.065777 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.065857 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 13:03:56.066200 3701 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42748
I20260926 13:03:56.072937 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:56.067100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.2.86.62:33523)
0926 13:03:56.067234 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:56.067236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:56.067244 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:56.067264 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:56.067264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:56.067281 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:56.067347 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:56.067828 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.068320 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.069086 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:56.069288 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:56.069371 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:56.070464 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:56.070465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:56.070466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:56.070473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:56.071802 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:56.072148 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:56.072149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:56.072150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:56.072209 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:56.072415 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:56.072416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:56.072416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:56.072519 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:56.072624 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:56.072714 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 13:03:56.072788 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
W20260926 13:03:56.073168 3502 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33838
I20260926 13:03:56.073650 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3576
I20260926 13:03:56.081404 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3716
I20260926 13:03:56.087446 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3856
I20260926 13:03:56.092034 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 3472
2026-09-26T13:03:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2716 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:03:56 dist-test-slave-j4lq krb5kdc[4088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:03:56 dist-test-slave-j4lq krb5kdc[4088](info): set up 2 sockets
Sep 26 13:03:56 dist-test-slave-j4lq krb5kdc[4088](info): commencing operation
krb5kdc: starting...
W20260926 13:03:57.112609 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:03:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:03:57.224041 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38889
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:57.301651 4101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:57.301877 4101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:57.301914 4101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:57.303313 4101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:03:57.303409 4101 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:03:57.303435 4101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:57.303467 4101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:57.303500 4101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:03:57.303522 4101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:03:57.305083 4101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:38889
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:57.305348 4101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:57.305563 4101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:57.310212 4107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:57.310190 4109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.310228 4101 server_base.cc:1034] running on GCE node
W20260926 13:03:57.310199 4106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.310638 4101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:57.310949 4101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:57.312106 4101 hybrid_clock.cc:648] HybridClock initialized: now 1790427837312093 us; error 46 us; skew 500 ppm
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:57.314463 4101 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:03:57.314944 4101 webserver.cc:460] Webserver started at http://127.2.86.62:44501/ using document root <none> and password file <none>
I20260926 13:03:57.315176 4101 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:57.315241 4101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:57.315390 4101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:57.316397 4101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "0346ebecc9e0497d8cdc7958b3a26b7e"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "3ba6eae57a7cdd43ebba8d2e77441cea"
server_key_iv: "0238388e1c1113e00bcd38965b630354"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.316759 4101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "0346ebecc9e0497d8cdc7958b3a26b7e"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "3ba6eae57a7cdd43ebba8d2e77441cea"
server_key_iv: "0238388e1c1113e00bcd38965b630354"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.318082 4101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:03:57.318773 4116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.318933 4101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:03:57.319034 4101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "0346ebecc9e0497d8cdc7958b3a26b7e"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "3ba6eae57a7cdd43ebba8d2e77441cea"
server_key_iv: "0238388e1c1113e00bcd38965b630354"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.319104 4101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:57.341212 4101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:57.343474 4101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:57.343628 4101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:57.348186 4101 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:38889
I20260926 13:03:57.348233 4178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:38889 every 8 connection(s)
I20260926 13:03:57.348558 4101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:03:57.349228 4179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:57.351421 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Bootstrap starting.
I20260926 13:03:57.351972 4179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:57.352245 4179 log.cc:824] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:57.352979 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: No bootstrap required, opened a new log
I20260926 13:03:57.355535 4179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } }
I20260926 13:03:57.355664 4179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:57.355684 4179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0346ebecc9e0497d8cdc7958b3a26b7e, State: Initialized, Role: FOLLOWER
I20260926 13:03:57.355850 4179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } }
I20260926 13:03:57.355921 4179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:03:57.355962 4179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:03:57.356010 4179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:57.356501 4179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } }
I20260926 13:03:57.356608 4179 leader_election.cc:302] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0346ebecc9e0497d8cdc7958b3a26b7e; no voters:
I20260926 13:03:57.356786 4179 leader_election.cc:288] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:03:57.356812 4182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:57.357026 4182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [term 1 LEADER]: Becoming Leader. State: Replica: 0346ebecc9e0497d8cdc7958b3a26b7e, State: Running, Role: LEADER
I20260926 13:03:57.357053 4179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:03:57.357116 4182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } }
I20260926 13:03:57.357472 4184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0346ebecc9e0497d8cdc7958b3a26b7e. Latest consensus state: current_term: 1 leader_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } } }
I20260926 13:03:57.357481 4183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0346ebecc9e0497d8cdc7958b3a26b7e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38889 } } }
I20260926 13:03:57.357584 4183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:57.357573 4184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e [sys.catalog]: This master's current role is: LEADER
I20260926 13:03:57.357796 4191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:03:57.358192 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4101
I20260926 13:03:57.358289 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:03:57.358309 4191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:03:57.358482 2392 external_mini_cluster.cc:1428] Setting key 118cc0cf5056f769c190a7045d6e36c0
I20260926 13:03:57.359889 4191 catalog_manager.cc:1347] Generated new cluster ID: 99849cb451be4285af674b04cf137ef7
I20260926 13:03:57.359941 4191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:03:57.368026 4191 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:03:57.368585 4191 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:03:57.372931 4191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Generated new TSK 0
I20260926 13:03:57.373121 4191 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:57.377907 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.358960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.2.86.62:38889)
0926 13:03:57.367739 (+ 8779us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.367747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.367771 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:03:57.367898 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.367909 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.367967 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.368115 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.368906 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.370263 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.371344 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.371952 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.372044 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.374464 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.374468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.374468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.374483 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.376051 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.376416 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.376419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.376419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.376919 (+ 500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.377128 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.377129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.377130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.377340 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.377498 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.377592 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.377744 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8736,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:03:57.409595 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38889
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:57.501271 4205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:57.501457 4205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:57.501479 4205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:57.502611 4205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:57.502657 4205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:57.502678 4205 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:03:57.502691 4205 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:57.502707 4205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:03:57.504165 4205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:38889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:57.504426 4205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:57.504638 4205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:57.509526 4210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:57.509534 4211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.509593 4205 server_base.cc:1034] running on GCE node
W20260926 13:03:57.509534 4213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.509912 4205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:57.510156 4205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:57.511300 4205 hybrid_clock.cc:648] HybridClock initialized: now 1790427837511285 us; error 39 us; skew 500 ppm
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:57.513657 4205 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:03:57.514113 4205 webserver.cc:460] Webserver started at http://127.2.86.1:44985/ using document root <none> and password file <none>
I20260926 13:03:57.514321 4205 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:57.514392 4205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:57.514539 4205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:57.515570 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "512b203f13cf4f3ba6e4330e91f7a621"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "ca72a770860c7ddb1933117ee157293b"
server_key_iv: "16fdf0774f741405eb4e1e9fdb00b309"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.515913 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "512b203f13cf4f3ba6e4330e91f7a621"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "ca72a770860c7ddb1933117ee157293b"
server_key_iv: "16fdf0774f741405eb4e1e9fdb00b309"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.517249 4205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:03:57.517942 4220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.518111 4205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:57.518204 4205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "512b203f13cf4f3ba6e4330e91f7a621"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "ca72a770860c7ddb1933117ee157293b"
server_key_iv: "16fdf0774f741405eb4e1e9fdb00b309"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.518276 4205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:57.534538 4205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:57.536211 4205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:57.536373 4205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:57.536839 4205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:57.536985 4205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.537029 4205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:57.537058 4205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:57.543984 4205 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:44663
I20260926 13:03:57.544116 4335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:44663 every 8 connection(s)
I20260926 13:03:57.544386 4205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:03:57.546837 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.537356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:58483 (local address 127.2.86.62:38889)
0926 13:03:57.537432 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.537434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.537662 (+ 228us) server_negotiation.cc:407] Connection header received
0926 13:03:57.538110 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.538112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.538127 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.538212 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.538859 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.539691 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.541159 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.541399 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.541481 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.544176 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.544178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.544178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.544186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.545742 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.546091 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.546092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.546092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.546140 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.546346 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.546346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.546346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.546457 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.546506 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.546714 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.546778 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:03:57.546983 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.537361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.1:58483)
0926 13:03:57.537534 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.537572 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.538013 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.538284 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.538288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.538557 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.538744 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.538748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.539811 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.539812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.541058 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.541061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.541501 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.541502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.541623 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.542180 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.542189 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.544078 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.545811 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.545817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.545818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.546025 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.546208 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.546209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.546209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.546295 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.546511 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.546514 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.546612 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.546686 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.546790 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260926 13:03:57.547463 4336 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38889
I20260926 13:03:57.547552 4336 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:57.547708 4336 heartbeater.cc:511] Master 127.2.86.62:38889 requested a full tablet report, sending...
I20260926 13:03:57.548223 4132 ts_manager.cc:194] Registered new tserver with Master: 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663)
I20260926 13:03:57.549113 4132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:58483
I20260926 13:03:57.553357 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.548497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:48723 (local address 127.2.86.62:38889)
0926 13:03:57.548546 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.548548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.548606 (+ 58us) server_negotiation.cc:407] Connection header received
0926 13:03:57.548681 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.548682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.548689 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.548731 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.549075 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.549696 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.550408 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.550609 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.550656 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.551499 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.551499 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.551500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.551504 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.552544 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.552817 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.552817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.552817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.552861 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.553045 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.553046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.553046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.553125 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.553167 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.553253 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.553301 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:03:57.553375 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.548405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.1:48723)
0926 13:03:57.548535 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.548546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.548612 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.548747 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.548748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.548956 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.549015 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.549017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.549811 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.549812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.550333 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.550335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.550669 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.550670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.550720 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.551140 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.551144 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.551434 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.552609 (+ 1175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.552610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.552610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.552756 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.552955 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.552955 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.552955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.553008 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.553159 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.553160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.553188 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.553241 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.553306 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260926 13:03:57.554448 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4205
I20260926 13:03:57.554569 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:03:57.554765 2392 external_mini_cluster.cc:1428] Setting key e0588d5aac2657f133193b54cb7d0311
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:03:57.579146 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38889
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:57.658237 4345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:57.658419 4345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:57.658442 4345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:57.659560 4345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:57.659600 4345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:57.659621 4345 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:03:57.659632 4345 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:57.659648 4345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:03:57.661046 4345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:38889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:57.661216 4345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:57.661366 4345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:57.665977 4353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.666028 4345 server_base.cc:1034] running on GCE node
W20260926 13:03:57.665985 4350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:57.665985 4351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.666370 4345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:57.666635 4345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:57.667795 4345 hybrid_clock.cc:648] HybridClock initialized: now 1790427837667772 us; error 38 us; skew 500 ppm
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:57.670156 4345 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:03:57.670605 4345 webserver.cc:460] Webserver started at http://127.2.86.2:34643/ using document root <none> and password file <none>
I20260926 13:03:57.670843 4345 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:57.670908 4345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:57.671048 4345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:57.672040 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "0dccbaffa54444a9989a2b34357bae75"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "fd79d568b224012d4a6bb1ac0f6a96c3"
server_key_iv: "d791994fb8d1a0cf071252f4775885a4"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.672382 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "0dccbaffa54444a9989a2b34357bae75"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "fd79d568b224012d4a6bb1ac0f6a96c3"
server_key_iv: "d791994fb8d1a0cf071252f4775885a4"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.673710 4345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:03:57.674459 4360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.674633 4345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:03:57.674727 4345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "0dccbaffa54444a9989a2b34357bae75"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "fd79d568b224012d4a6bb1ac0f6a96c3"
server_key_iv: "d791994fb8d1a0cf071252f4775885a4"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.674801 4345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:57.687114 4345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:57.688716 4345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:57.688900 4345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:57.689349 4345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:57.689409 4345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.689447 4345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:57.689477 4345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.696144 4345 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:32933
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:57.696269 4475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:32933 every 8 connection(s)
I20260926 13:03:57.696506 4345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:03:57.699679 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.689842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:48799 (local address 127.2.86.62:38889)
0926 13:03:57.689902 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.689904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.690096 (+ 192us) server_negotiation.cc:407] Connection header received
0926 13:03:57.690515 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.690516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.690530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.690623 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.691418 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.692244 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.693773 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.694141 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.694223 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.696847 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.696848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.696849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.696856 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.698314 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.698733 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.698735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.698735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.698808 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.699064 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.699065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.699065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.699176 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.699251 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.699517 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.699594 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:03:57.699843 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.689844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.2:48799)
0926 13:03:57.689994 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.690025 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.690415 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.690684 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.690687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.691045 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.691306 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.691312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.692336 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.692336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.693676 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.693679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.694237 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.694238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.694353 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.694894 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.694905 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.696660 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.698418 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.698424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.698426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.698641 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.698890 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.698891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.698891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.698997 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.699243 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.699248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.699390 (+ 142us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.699494 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.699635 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260926 13:03:57.700353 4476 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38889
I20260926 13:03:57.700429 4476 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:57.700596 4476 heartbeater.cc:511] Master 127.2.86.62:38889 requested a full tablet report, sending...
I20260926 13:03:57.700950 4132 ts_manager.cc:194] Registered new tserver with Master: 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933)
I20260926 13:03:57.701422 4132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:48799
I20260926 13:03:57.703476 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4345
I20260926 13:03:57.703578 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:03:57.703755 2392 external_mini_cluster.cc:1428] Setting key d753ff42980e2b0760419b862540bce9
I20260926 13:03:57.707037 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.701343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:41627 (local address 127.2.86.62:38889)
0926 13:03:57.701383 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.701385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.701439 (+ 54us) server_negotiation.cc:407] Connection header received
0926 13:03:57.701520 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.701521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.701530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.701584 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.701885 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.702357 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.703053 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.703257 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.703335 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.704284 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.704285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.704286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.704292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.705848 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.706229 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.706230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.706231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.706282 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.706506 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.706507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.706508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.706645 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.706723 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.706883 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.706953 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:03:57.707054 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.701246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.2:41627)
0926 13:03:57.701377 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.701387 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.701457 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.701591 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.701592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.701746 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.701790 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.701791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.702449 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.702450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.702959 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.702960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.703340 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.703341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.703412 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.703839 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.703844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.704208 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.705969 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.705970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.705970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.706151 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.706377 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.706378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.706378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.706434 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.706715 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.706716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.706758 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.706858 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.706953 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:03:57.727741 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38889
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:03:57.810434 4485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:03:57.810665 4485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:03:57.810705 4485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:03:57.811873 4485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:03:57.811940 4485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:03:57.811977 4485 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 13:03:57.812016 4485 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:03:57.812054 4485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:03:57.813560 4485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:38889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:03:57.813802 4485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:03:57.814023 4485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:03:57.818850 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:03:57.818832 4491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.818923 4485 server_base.cc:1034] running on GCE node
W20260926 13:03:57.819082 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:03:57.819303 4485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:03:57.819515 4485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:03:57.820657 4485 hybrid_clock.cc:648] HybridClock initialized: now 1790427837820638 us; error 32 us; skew 500 ppm
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:03:57.829155 4485 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:03:57.829610 4485 webserver.cc:460] Webserver started at http://127.2.86.3:41579/ using document root <none> and password file <none>
I20260926 13:03:57.829847 4485 fs_manager.cc:359] Metadata directory not provided
I20260926 13:03:57.829891 4485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:03:57.830039 4485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:03:57.831056 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "2f9c50ca5f20474abe8786996cd81118"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "f984eccc23f4478a31be4dac46f1a1e7"
server_key_iv: "d2f24e67477724a812ce47625452d9ab"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.831396 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9c50ca5f20474abe8786996cd81118"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "f984eccc23f4478a31be4dac46f1a1e7"
server_key_iv: "d2f24e67477724a812ce47625452d9ab"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.832769 4485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:03:57.833621 4500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.833854 4485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:03:57.833956 4485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "2f9c50ca5f20474abe8786996cd81118"
format_stamp: "Formatted at 2026-09-26 13:03:57 on dist-test-slave-j4lq"
server_key: "f984eccc23f4478a31be4dac46f1a1e7"
server_key_iv: "d2f24e67477724a812ce47625452d9ab"
server_key_version: "encryptionkey@0"
I20260926 13:03:57.834038 4485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:03:57.847856 4485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:03:57.850169 4485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:03:57.850319 4485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:03:57.850873 4485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:03:57.850930 4485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:57.850973 4485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:03:57.850996 4485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:57.858248 4485 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38607
I20260926 13:03:57.858325 4615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38607 every 8 connection(s)
I20260926 13:03:57.858640 4485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:03:57.860677 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.851259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:48985 (local address 127.2.86.62:38889)
0926 13:03:57.851340 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.851342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.851642 (+ 300us) server_negotiation.cc:407] Connection header received
0926 13:03:57.852306 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.852307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.852333 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.852406 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.853313 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.854125 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.855364 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.855576 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.855655 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.858124 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.858126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.858127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.858135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.859627 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.859971 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.859972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.859972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.860018 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.860177 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.860178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.860178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.860264 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.860307 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.860558 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.860617 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:03:57.860836 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.851270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.3:48985)
0926 13:03:57.851477 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.851529 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.852196 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.852487 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.852492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.852859 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.853212 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.853219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.854216 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.854217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.855262 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.855265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.855672 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.855673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.855789 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.856395 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.856404 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.858036 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.859695 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.859698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.859700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.859896 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.860058 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.860058 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.860058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.860134 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.860326 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.860330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.860449 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.860526 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.860633 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260926 13:03:57.861267 4616 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38889
I20260926 13:03:57.861354 4616 heartbeater.cc:464] Registering TS with master...
I20260926 13:03:57.861531 4616 heartbeater.cc:511] Master 127.2.86.62:38889 requested a full tablet report, sending...
I20260926 13:03:57.861860 4133 ts_manager.cc:194] Registered new tserver with Master: 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:03:57.862372 4133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:48985
I20260926 13:03:57.862757 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4485
I20260926 13:03:57.862865 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:03:57.863075 2392 external_mini_cluster.cc:1428] Setting key d3aec6e609de6da01b9467866cdb8bcd
I20260926 13:03:57.864035 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:03:57.868992 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.862385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:59355 (local address 127.2.86.62:38889)
0926 13:03:57.862422 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.862423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.863290 (+ 867us) server_negotiation.cc:407] Connection header received
0926 13:03:57.863361 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.863361 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.863369 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.863408 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.863782 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.864352 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.865154 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.865356 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.865444 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.866311 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.866313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.866313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.866321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.867875 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.868206 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.868207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.868208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.868275 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.868454 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.868456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.868456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.868586 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.868662 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.868761 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.868847 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:03:57.868996 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.862266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.2.86.3:59355)
0926 13:03:57.863206 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.863217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.863283 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.863411 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.863412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.863641 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:57.863708 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.863710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.864442 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.864443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.865058 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.865060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.865446 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.865447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.865513 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.865949 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:57.865954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:57.866217 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:57.867963 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.867964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.867965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.868130 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.868344 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:57.868345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:57.868345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:57.868392 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:57.868656 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:57.868656 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:57.868688 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.868737 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.868832 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:03:57 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:57.890827 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.883519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.2.86.62:38889)
0926 13:03:57.883555 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.883557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.883612 (+ 55us) server_negotiation.cc:407] Connection header received
0926 13:03:57.883709 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.883709 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.883718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.883800 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:57.884489 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.885198 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.886294 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.886489 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.886569 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.888602 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:57.888604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:57.888604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:57.888611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:57.889734 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.890070 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.890071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.890071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.890124 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:57.890324 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:57.890324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:57.890325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:57.890448 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:57.890551 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.890669 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.890735 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:03:57.892273 4133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:03:57.893901 4133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:57.899938 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.0.0.1:51392)
0926 13:03:57.896263 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.896299 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.896373 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.896634 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.896638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.896640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:57.896695 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.896697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.897723 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.897724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.898754 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.898756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.899639 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.899640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.899724 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.899737 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.899811 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.899861 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260926 13:03:57.899955 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.2.86.3:38607)
0926 13:03:57.896331 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.896336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.896346 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:03:57.896484 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.896486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.896502 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.896568 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:57.896805 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.897639 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.898869 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.899558 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.899640 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.899696 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.899830 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.899870 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260926 13:03:57.900427 4634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32933 (local address 127.0.0.1:51018)
0926 13:03:57.896648 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.896657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.896712 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.897075 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.897076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.897078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:57.897115 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.897116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.898312 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.898313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.899203 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.899205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.900169 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.900170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.900258 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.900263 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.900326 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.900361 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":62,"threads_started":1}
I20260926 13:03:57.900477 4637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.2.86.2:32933)
0926 13:03:57.896639 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.896643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.896702 (+ 59us) server_negotiation.cc:407] Connection header received
0926 13:03:57.896922 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.896925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.896940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.897013 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:57.897197 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.898246 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.899292 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.900002 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.900079 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.900136 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.900348 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.900390 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260926 13:03:57.900656 4550 tablet_service.cc:1467] Processing CreateTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 (DEFAULT_TABLE table=test-table [id=72dc05f621e84e3aa6477db50a33d4eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:57.901068 4550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9f289c5b6e406684c616a6ff21c4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:57.901106 4410 tablet_service.cc:1467] Processing CreateTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 (DEFAULT_TABLE table=test-table [id=72dc05f621e84e3aa6477db50a33d4eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:57.901221 4636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:44663 (local address 127.0.0.1:57770)
0926 13:03:57.896446 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:57.896456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:57.896505 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:57.896835 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:57.896836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:57.896838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:57.896910 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.896912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.898250 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.898251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.899362 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:57.899364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.900825 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.900826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:57.901045 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.901050 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:03:57.901102 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:03:57.901143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":29,"thread_start_us":95,"threads_started":1}
I20260926 13:03:57.901436 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9f289c5b6e406684c616a6ff21c4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:57.901510 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:57.896484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.2.86.1:44663)
0926 13:03:57.896623 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:57.896628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:57.896638 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:03:57.896679 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:57.896681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:57.896696 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:57.896989 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:57.897055 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.898148 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.899444 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:57.900718 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:57.900822 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:57.900953 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:57.901365 (+ 412us) server_negotiation.cc:299] Negotiation successful
0926 13:03:57.901415 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260926 13:03:57.904338 4639 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Bootstrap starting.
I20260926 13:03:57.904713 4270 tablet_service.cc:1467] Processing CreateTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 (DEFAULT_TABLE table=test-table [id=72dc05f621e84e3aa6477db50a33d4eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:57.904915 4640 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Bootstrap starting.
I20260926 13:03:57.905092 4270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9f289c5b6e406684c616a6ff21c4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:57.905587 4640 tablet_bootstrap.cc:654] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:57.905587 4639 tablet_bootstrap.cc:654] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:57.905937 4639 log.cc:824] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:57.905937 4640 log.cc:824] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:57.906738 4640 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: No bootstrap required, opened a new log
I20260926 13:03:57.906801 4639 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: No bootstrap required, opened a new log
I20260926 13:03:57.906836 4640 ts_tablet_manager.cc:1397] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:57.906869 4639 ts_tablet_manager.cc:1397] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:03:57.908507 4643 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Bootstrap starting.
I20260926 13:03:57.908608 4640 raft_consensus.cc:348] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.908771 4640 raft_consensus.cc:374] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:57.908838 4640 raft_consensus.cc:729] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Initialized, Role: FOLLOWER
I20260926 13:03:57.909006 4640 consensus_queue.cc:260] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.908978 4639 raft_consensus.cc:348] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.909149 4639 raft_consensus.cc:374] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:57.909185 4639 raft_consensus.cc:729] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Initialized, Role: FOLLOWER
I20260926 13:03:57.909220 4643 tablet_bootstrap.cc:654] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:57.909277 4640 ts_tablet_manager.cc:1428] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:03:57.909327 4639 consensus_queue.cc:260] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.909543 4643 log.cc:824] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Log is configured to *not* fsync() on all Append() calls
I20260926 13:03:57.909569 4476 heartbeater.cc:503] Master 127.2.86.62:38889 was elected leader, sending a full tablet report...
I20260926 13:03:57.909581 4639 ts_tablet_manager.cc:1428] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 13:03:57.909592 4616 heartbeater.cc:503] Master 127.2.86.62:38889 was elected leader, sending a full tablet report...
I20260926 13:03:57.910452 4643 tablet_bootstrap.cc:492] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: No bootstrap required, opened a new log
I20260926 13:03:57.910524 4643 ts_tablet_manager.cc:1397] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:03:57.911996 4643 raft_consensus.cc:348] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.912110 4643 raft_consensus.cc:374] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:57.912130 4643 raft_consensus.cc:729] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Initialized, Role: FOLLOWER
I20260926 13:03:57.912259 4643 consensus_queue.cc:260] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:57.912501 4643 ts_tablet_manager.cc:1428] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:03:57.912554 4336 heartbeater.cc:503] Master 127.2.86.62:38889 was elected leader, sending a full tablet report...
W20260926 13:03:57.947130 4477 tablet.cc:2369] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:03:58.045426 4337 tablet.cc:2369] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:03:58.109184 4617 tablet.cc:2369] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:03:58.247026 4644 raft_consensus.cc:482] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:58.247156 4644 raft_consensus.cc:504] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:58.247651 4644 leader_election.cc:288] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:03:58.250452 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.247900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:38819 (local address 127.2.86.1:44663)
0926 13:03:58.247949 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.247952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.248054 (+ 102us) server_negotiation.cc:407] Connection header received
0926 13:03:58.248122 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.248123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.248135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.248182 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.248318 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.248841 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.249770 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.250117 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.250189 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.250200 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.250357 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.250396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:03:58.250466 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.247858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:44663 (local address 127.2.86.2:38819)
0926 13:03:58.247981 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.247990 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.248056 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.248185 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.248186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.248188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.248249 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.248251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.248957 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.248958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.249675 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.249677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.250201 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.250202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.250266 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.250270 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.250341 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.250380 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260926 13:03:58.250512 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.247892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:44885 (local address 127.2.86.3:38607)
0926 13:03:58.247935 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.247938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.248054 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:03:58.248119 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.248120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.248132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.248201 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.248361 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.248944 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.249822 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.250163 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.250236 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.250246 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.250406 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.250442 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:03:58.250548 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.247812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.2.86.2:44885)
0926 13:03:58.247973 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.247984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.248064 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.248215 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.248216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.248218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.248283 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.248284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.249040 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.249041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.249721 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.249723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.250253 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.250254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.250321 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.250327 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.250390 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.250429 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":64,"threads_started":1}
I20260926 13:03:58.250885 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9f289c5b6e406684c616a6ff21c4f5" candidate_uuid: "0dccbaffa54444a9989a2b34357bae75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621" is_pre_election: true
I20260926 13:03:58.251039 4290 raft_consensus.cc:2454] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dccbaffa54444a9989a2b34357bae75 in term 0.
I20260926 13:03:58.251019 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9f289c5b6e406684c616a6ff21c4f5" candidate_uuid: "0dccbaffa54444a9989a2b34357bae75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118" is_pre_election: true
I20260926 13:03:58.251138 4570 raft_consensus.cc:2454] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dccbaffa54444a9989a2b34357bae75 in term 0.
I20260926 13:03:58.251343 4361 leader_election.cc:302] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 2f9c50ca5f20474abe8786996cd81118; no voters:
I20260926 13:03:58.251482 4644 raft_consensus.cc:2781] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:58.251533 4644 raft_consensus.cc:482] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:58.251583 4644 raft_consensus.cc:3037] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.252278 4644 raft_consensus.cc:504] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:58.252415 4644 leader_election.cc:288] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [CANDIDATE]: Term 1 election: Requested vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:03:58.252545 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9f289c5b6e406684c616a6ff21c4f5" candidate_uuid: "0dccbaffa54444a9989a2b34357bae75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621"
I20260926 13:03:58.252552 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9f289c5b6e406684c616a6ff21c4f5" candidate_uuid: "0dccbaffa54444a9989a2b34357bae75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118"
I20260926 13:03:58.252643 4290 raft_consensus.cc:3037] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.252638 4570 raft_consensus.cc:3037] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.253428 4570 raft_consensus.cc:2454] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dccbaffa54444a9989a2b34357bae75 in term 1.
I20260926 13:03:58.253521 4290 raft_consensus.cc:2454] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dccbaffa54444a9989a2b34357bae75 in term 1.
I20260926 13:03:58.253587 4361 leader_election.cc:302] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 2f9c50ca5f20474abe8786996cd81118; no voters:
I20260926 13:03:58.253661 4644 raft_consensus.cc:2781] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:58.253764 4644 raft_consensus.cc:686] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 1 LEADER]: Becoming Leader. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Running, Role: LEADER
I20260926 13:03:58.253860 4644 consensus_queue.cc:237] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } }
I20260926 13:03:58.254630 4132 catalog_manager.cc:5495] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "0dccbaffa54444a9989a2b34357bae75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } health_report { overall_health: HEALTHY } } }
Sep 26 13:03:58 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427837, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:03:58.355960 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.349483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38889 (local address 127.0.0.1:35276)
0926 13:03:58.349654 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.349663 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.349740 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.349890 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.349891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.350181 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:03:58.350221 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.350223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.350881 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.350882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.351401 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.351403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.351767 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.351768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.351815 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.352351 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:03:58.352355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:03:58.353754 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:03:58.355091 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:58.355097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:58.355099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:58.355286 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:58.355468 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:03:58.355469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:03:58.355469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:03:58.355522 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:03:58.355717 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:03:58.355721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:03:58.355763 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.355814 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.355897 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260926 13:03:58.356056 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.349546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.2.86.62:38889)
0926 13:03:58.349606 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.349609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.349723 (+ 114us) server_negotiation.cc:407] Connection header received
0926 13:03:58.349801 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.349802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.349811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.349887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:03:58.350305 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.350779 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.351485 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.351668 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.351769 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.353831 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:03:58.353832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:03:58.353833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:03:58.353844 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:03:58.355005 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:58.355355 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:58.355355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:58.355356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:58.355411 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:03:58.355573 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:03:58.355574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:03:58.355574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:03:58.355666 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:03:58.355714 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.355852 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.355941 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":36}
I20260926 13:03:58.358093 4132 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:35276:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:03:58.358340 4132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:58.359869 4270 tablet_service.cc:1467] Processing CreateTablet for tablet 894f32977d874210868ff02fc21eb366 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c787bb3d00124543844410c7307e9f57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:58.359874 4410 tablet_service.cc:1467] Processing CreateTablet for tablet 894f32977d874210868ff02fc21eb366 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c787bb3d00124543844410c7307e9f57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:58.360028 4270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f32977d874210868ff02fc21eb366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.360037 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f32977d874210868ff02fc21eb366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.361318 4643 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: Bootstrap starting.
I20260926 13:03:58.361506 4550 tablet_service.cc:1467] Processing CreateTablet for tablet 894f32977d874210868ff02fc21eb366 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c787bb3d00124543844410c7307e9f57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:03:58.361644 4550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 894f32977d874210868ff02fc21eb366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.361732 4643 tablet_bootstrap.cc:654] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.362390 4643 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: No bootstrap required, opened a new log
I20260926 13:03:58.362448 4643 ts_tablet_manager.cc:1397] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:58.362607 4643 raft_consensus.cc:348] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.362680 4643 raft_consensus.cc:374] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.362699 4643 raft_consensus.cc:729] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.362739 4643 consensus_queue.cc:260] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.362850 4643 ts_tablet_manager.cc:1428] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:03:58.363008 4647 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.363095 4647 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.363196 4639 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: Bootstrap starting.
I20260926 13:03:58.363300 4640 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: Bootstrap starting.
I20260926 13:03:58.363677 4639 tablet_bootstrap.cc:654] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.363783 4640 tablet_bootstrap.cc:654] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.364329 4639 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: No bootstrap required, opened a new log
I20260926 13:03:58.364387 4639 ts_tablet_manager.cc:1397] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:03:58.364533 4640 tablet_bootstrap.cc:492] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: No bootstrap required, opened a new log
I20260926 13:03:58.364594 4640 ts_tablet_manager.cc:1397] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:58.364656 4639 raft_consensus.cc:348] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.364748 4639 raft_consensus.cc:374] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.364768 4639 raft_consensus.cc:729] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.364817 4639 consensus_queue.cc:260] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.364949 4639 ts_tablet_manager.cc:1428] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:58.364928 4640 raft_consensus.cc:348] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.364993 4640 raft_consensus.cc:374] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.365031 4640 raft_consensus.cc:729] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.365079 4640 consensus_queue.cc:260] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.365079 4645 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.365170 4640 ts_tablet_manager.cc:1428] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:58.365162 4645 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.365295 4644 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.365378 4644 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.462951 4645 raft_consensus.cc:482] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:58.463047 4645 raft_consensus.cc:504] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.463414 4645 leader_election.cc:288] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933)
I20260926 13:03:58.467471 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.463881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34601 (local address 127.2.86.2:32933)
0926 13:03:58.464041 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.464043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.464050 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:03:58.464075 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.464076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.464085 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.464154 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.464298 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.465038 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.466560 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.467119 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.467183 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.467194 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.467361 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.467398 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260926 13:03:58.467470 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.463705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:37141 (local address 127.2.86.1:44663)
0926 13:03:58.463772 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.463775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.463827 (+ 52us) server_negotiation.cc:407] Connection header received
0926 13:03:58.463916 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.463917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.463926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.464014 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.464216 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.465011 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.466563 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.467107 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.467183 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.467194 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.467358 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.467398 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:03:58.467495 4661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.463596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:44663 (local address 127.2.86.3:37141)
0926 13:03:58.463759 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.463772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.463846 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.464046 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.464047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.464049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.464119 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.464120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.465112 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.465113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.466471 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.466473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.467200 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.467202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.467270 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.467275 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.467335 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.467379 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":64,"thread_start_us":91,"threads_started":1}
I20260926 13:03:58.467497 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.463789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32933 (local address 127.2.86.3:34601)
0926 13:03:58.463887 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.463899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.463953 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.464181 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.464182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.464184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.464226 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.464228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.465112 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.465113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.466470 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.466473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.467220 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.467221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.467282 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.467285 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.467335 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.467379 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":87,"thread_start_us":37,"threads_started":1}
I20260926 13:03:58.467774 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f32977d874210868ff02fc21eb366" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621" is_pre_election: true
I20260926 13:03:58.467867 4290 raft_consensus.cc:2454] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 0.
I20260926 13:03:58.468015 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f32977d874210868ff02fc21eb366" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75" is_pre_election: true
I20260926 13:03:58.468052 4504 leader_election.cc:302] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c50ca5f20474abe8786996cd81118, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:03:58.468146 4430 raft_consensus.cc:2454] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 0.
I20260926 13:03:58.468180 4645 raft_consensus.cc:2781] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:58.468225 4645 raft_consensus.cc:482] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:58.468259 4645 raft_consensus.cc:3037] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.468990 4645 raft_consensus.cc:504] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.469121 4645 leader_election.cc:288] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 election: Requested vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933)
I20260926 13:03:58.469301 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f32977d874210868ff02fc21eb366" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75"
I20260926 13:03:58.469290 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "894f32977d874210868ff02fc21eb366" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621"
I20260926 13:03:58.469370 4290 raft_consensus.cc:3037] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.469370 4430 raft_consensus.cc:3037] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.470127 4430 raft_consensus.cc:2454] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 1.
I20260926 13:03:58.470127 4290 raft_consensus.cc:2454] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 1.
I20260926 13:03:58.470317 4504 leader_election.cc:302] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c50ca5f20474abe8786996cd81118, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:03:58.470400 4645 raft_consensus.cc:2781] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:58.470511 4645 raft_consensus.cc:686] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Running, Role: LEADER
I20260926 13:03:58.470638 4645 consensus_queue.cc:237] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.471063 4664 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: TxnStatusTablet state changed. Reason: New leader 2f9c50ca5f20474abe8786996cd81118. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.471158 4664 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:58.471321 4666 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:03:58.471452 4666 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:03:58.471379 4132 catalog_manager.cc:5495] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260926 13:03:58.481727 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.479301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51400 (local address 127.2.86.3:38607)
0926 13:03:58.479364 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.479366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.479375 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:03:58.479460 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.479461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.479476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.479549 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.479723 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.480200 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.480967 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.481165 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.481236 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.481416 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.481510 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.481587 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.481651 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.481670 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:03:58.481763 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.479250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.0.0.1:51400)
0926 13:03:58.479324 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.479334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.479382 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.479593 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.479594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.479596 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:58.479644 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.479645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.480289 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.480290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.480842 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.480844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.481235 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.481236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.481291 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.481305 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.481579 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.481586 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.481646 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.481695 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 13:03:58.483368 4430 raft_consensus.cc:1261] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Refusing update from remote peer 2f9c50ca5f20474abe8786996cd81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.483409 4290 raft_consensus.cc:1261] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Refusing update from remote peer 2f9c50ca5f20474abe8786996cd81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.483646 4664 consensus_queue.cc:1035] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.483771 4645 consensus_queue.cc:1035] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.484792 4644 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: TxnStatusTablet state changed. Reason: New leader 2f9c50ca5f20474abe8786996cd81118. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.484808 4647 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: TxnStatusTablet state changed. Reason: New leader 2f9c50ca5f20474abe8786996cd81118. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.484905 4644 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.484920 4647 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.485839 4645 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.485920 4645 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:58.485991 4669 mvcc.cc:205] Tried to move back new op lower bound from 7333592426425548800 to 7333592426376331264. Current Snapshot: MvccSnapshot[applied={T|T < 7333592426425548800}]
I20260926 13:03:58.486137 4647 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.486218 4647 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 512b203f13cf4f3ba6e4330e91f7a621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.486502 4664 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.486578 4664 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 2f9c50ca5f20474abe8786996cd81118: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:03:58.486778 4671 mvcc.cc:205] Tried to move back new op lower bound from 7333592426425548800 to 7333592426376331264. Current Snapshot: MvccSnapshot[applied={T|T < 7333592426425548800 or (T in {7333592426425548800})}]
I20260926 13:03:58.487030 4670 mvcc.cc:205] Tried to move back new op lower bound from 7333592426425548800 to 7333592426376331264. Current Snapshot: MvccSnapshot[applied={T|T < 7333592426425548800}]
I20260926 13:03:58.487087 4644 tablet_replica.cc:439] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } } }
I20260926 13:03:58.487151 4644 tablet_replica.cc:442] T 894f32977d874210868ff02fc21eb366 P 0dccbaffa54444a9989a2b34357bae75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:03:58.491223 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.488802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.2.86.2:32933)
0926 13:03:58.488835 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.488837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.488997 (+ 160us) server_negotiation.cc:407] Connection header received
0926 13:03:58.489087 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.489088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.489104 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.489163 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.489279 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.489711 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.490472 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.490695 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.490765 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.490936 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.491020 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.491103 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.491139 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.491157 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:03:58.495705 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.493494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:34323 (local address 127.2.86.3:38607)
0926 13:03:58.493533 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.493535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.493601 (+ 66us) server_negotiation.cc:407] Connection header received
0926 13:03:58.493669 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.493670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.493679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.493730 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.493856 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.494276 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.495010 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.495226 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.495298 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.495478 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.495513 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.495569 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.495628 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.495644 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:58.495751 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.493423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.2.86.2:34323)
0926 13:03:58.493551 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.493560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.493615 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.493734 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.493734 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.493736 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:58.493795 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.493796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.494361 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.494362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.494931 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.494933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.495297 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.495297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.495363 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.495382 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.495570 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.495572 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.495624 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.495662 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260926 13:03:58.498193 4290 raft_consensus.cc:1261] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Refusing update from remote peer 0dccbaffa54444a9989a2b34357bae75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.498308 4570 raft_consensus.cc:1261] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Refusing update from remote peer 0dccbaffa54444a9989a2b34357bae75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.498425 4644 consensus_queue.cc:1035] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.498526 4650 consensus_queue.cc:1035] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.508211 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.505351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:36737 (local address 127.2.86.2:32933)
0926 13:03:58.505400 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.505402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.505558 (+ 156us) server_negotiation.cc:407] Connection header received
0926 13:03:58.505604 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.505605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.505613 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.505660 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.505834 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.506562 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.507311 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.507630 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.507698 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.507909 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.507940 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.507993 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.508116 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.508140 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 13:03:58.508452 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.505295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32933 (local address 127.2.86.3:36737)
0926 13:03:58.505461 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.505470 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.505548 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.505697 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.505698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.505699 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:58.505762 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.505764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.506642 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.506643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.507234 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.507236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.507727 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.507728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.507797 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.507811 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.508035 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.508036 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.508297 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.508361 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260926 13:03:58.515622 4133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:58.515719 4133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:03:58.516353 4133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Sending DeleteTablet for 3 replicas of tablet fb9f289c5b6e406684c616a6ff21c4f5
I20260926 13:03:58.516628 4550 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:58 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51392
I20260926 13:03:58.516638 4410 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:58 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51018
I20260926 13:03:58.516844 4270 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9f289c5b6e406684c616a6ff21c4f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:03:58 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:57770
I20260926 13:03:58.516978 4133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 13:03:58.517094 4692 tablet_replica.cc:330] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: stopping tablet replica
W20260926 13:03:58.517109 4133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:03:58.517195 4692 raft_consensus.cc:2229] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:58.517251 4692 raft_consensus.cc:2258] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:58.517493 4691 tablet_replica.cc:330] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: stopping tablet replica
I20260926 13:03:58.517581 4692 ts_tablet_manager.cc:1905] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:58.517668 4691 raft_consensus.cc:2229] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:03:58.517758 4691 raft_consensus.cc:2258] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:58.518162 4691 ts_tablet_manager.cc:1905] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:58.518333 4550 tablet_service.cc:1467] Processing CreateTablet for tablet 923f4440db24484e96d36e836f1ae78e (DEFAULT_TABLE table=test-table [id=632b44a92ee3437482c2955a1deca8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:58.518451 4550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923f4440db24484e96d36e836f1ae78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.519629 4692 ts_tablet_manager.cc:1918] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:58.519685 4639 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Bootstrap starting.
I20260926 13:03:58.519708 4692 log.cc:1196] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/fb9f289c5b6e406684c616a6ff21c4f5
I20260926 13:03:58.519977 4692 ts_tablet_manager.cc:1939] T fb9f289c5b6e406684c616a6ff21c4f5 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting consensus metadata
I20260926 13:03:58.520087 4639 tablet_bootstrap.cc:654] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.520572 4120 catalog_manager.cc:4841] TS 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663): tablet fb9f289c5b6e406684c616a6ff21c4f5 (table test-table [id=72dc05f621e84e3aa6477db50a33d4eb]) successfully deleted
I20260926 13:03:58.520675 4691 ts_tablet_manager.cc:1918] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:58.520744 4691 log.cc:1196] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/fb9f289c5b6e406684c616a6ff21c4f5
I20260926 13:03:58.520738 4410 tablet_service.cc:1467] Processing CreateTablet for tablet 923f4440db24484e96d36e836f1ae78e (DEFAULT_TABLE table=test-table [id=632b44a92ee3437482c2955a1deca8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:58.520849 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923f4440db24484e96d36e836f1ae78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.521051 4691 ts_tablet_manager.cc:1939] T fb9f289c5b6e406684c616a6ff21c4f5 P 2f9c50ca5f20474abe8786996cd81118: Deleting consensus metadata
I20260926 13:03:58.521440 4117 catalog_manager.cc:4841] TS 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607): tablet fb9f289c5b6e406684c616a6ff21c4f5 (table test-table [id=72dc05f621e84e3aa6477db50a33d4eb]) successfully deleted
I20260926 13:03:58.517943 4690 tablet_replica.cc:330] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: stopping tablet replica
I20260926 13:03:58.521613 4690 raft_consensus.cc:2229] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:03:58.521713 4690 raft_consensus.cc:2258] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:03:58.522045 4690 ts_tablet_manager.cc:1905] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:03:58.522189 4640 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Bootstrap starting.
I20260926 13:03:58.522567 4640 tablet_bootstrap.cc:654] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.523049 4270 tablet_service.cc:1467] Processing CreateTablet for tablet 923f4440db24484e96d36e836f1ae78e (DEFAULT_TABLE table=test-table [id=632b44a92ee3437482c2955a1deca8f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:03:58.523149 4270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923f4440db24484e96d36e836f1ae78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:03:58.523578 4690 ts_tablet_manager.cc:1918] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:03:58.523649 4690 log.cc:1196] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/fb9f289c5b6e406684c616a6ff21c4f5
I20260926 13:03:58.523911 4690 ts_tablet_manager.cc:1939] T fb9f289c5b6e406684c616a6ff21c4f5 P 0dccbaffa54444a9989a2b34357bae75: Deleting consensus metadata
I20260926 13:03:58.524256 4643 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Bootstrap starting.
I20260926 13:03:58.524642 4643 tablet_bootstrap.cc:654] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Neither blocks nor log segments found. Creating new log.
I20260926 13:03:58.525102 4120 catalog_manager.cc:4841] TS 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933): tablet fb9f289c5b6e406684c616a6ff21c4f5 (table test-table [id=72dc05f621e84e3aa6477db50a33d4eb]) successfully deleted
I20260926 13:03:58.525261 4643 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: No bootstrap required, opened a new log
I20260926 13:03:58.525303 4643 ts_tablet_manager.cc:1397] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:03:58.525430 4643 raft_consensus.cc:348] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.525467 4639 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: No bootstrap required, opened a new log
I20260926 13:03:58.525492 4643 raft_consensus.cc:374] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.525514 4643 raft_consensus.cc:729] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.525508 4639 ts_tablet_manager.cc:1397] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 13:03:58.525557 4643 consensus_queue.cc:260] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.525642 4643 ts_tablet_manager.cc:1428] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:58.525632 4639 raft_consensus.cc:348] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.525689 4639 raft_consensus.cc:374] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.525710 4639 raft_consensus.cc:729] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.525749 4639 consensus_queue.cc:260] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.525827 4639 ts_tablet_manager.cc:1428] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:58.526273 4640 tablet_bootstrap.cc:492] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: No bootstrap required, opened a new log
I20260926 13:03:58.526329 4640 ts_tablet_manager.cc:1397] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 13:03:58.526458 4640 raft_consensus.cc:348] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.526523 4640 raft_consensus.cc:374] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:03:58.526543 4640 raft_consensus.cc:729] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Initialized, Role: FOLLOWER
I20260926 13:03:58.526580 4640 consensus_queue.cc:260] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.526674 4640 ts_tablet_manager.cc:1428] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:03:58.639256 4647 raft_consensus.cc:482] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:03:58.639353 4647 raft_consensus.cc:504] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.639621 4647 leader_election.cc:288] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:03:58.642714 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.639923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:46399 (local address 127.2.86.2:32933)
0926 13:03:58.639968 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.639970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.640147 (+ 177us) server_negotiation.cc:407] Connection header received
0926 13:03:58.640220 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.640221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.640230 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.640275 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.640391 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.640839 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642006 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.642351 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642433 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.642444 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.642606 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.642637 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:03:58.642724 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.639934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41855 (local address 127.2.86.3:38607)
0926 13:03:58.639999 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.640001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.640151 (+ 150us) server_negotiation.cc:407] Connection header received
0926 13:03:58.640222 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.640223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.640232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.640286 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:03:58.640431 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.640891 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642003 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.642352 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642435 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.642445 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.642622 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.642653 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:03:58.642768 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.639856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32933 (local address 127.2.86.1:46399)
0926 13:03:58.640052 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.640063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.640154 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.640278 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.640278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.640280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.640336 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.640338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.640950 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.640951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.641912 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.641915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.642434 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.642501 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.642505 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.642600 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.642637 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":43,"thread_start_us":105,"threads_started":1}
I20260926 13:03:58.642943 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.639856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.2.86.1:41855)
0926 13:03:58.640052 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.640062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.640162 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.640295 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.640296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.640298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:03:58.640356 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.640358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.641019 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.641020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.641904 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.641906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.642438 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.642439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.642545 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.642549 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.642615 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.642656 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260926 13:03:58.642994 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923f4440db24484e96d36e836f1ae78e" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75" is_pre_election: true
I20260926 13:03:58.643091 4430 raft_consensus.cc:2454] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 0.
I20260926 13:03:58.643160 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923f4440db24484e96d36e836f1ae78e" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118" is_pre_election: true
I20260926 13:03:58.643229 4570 raft_consensus.cc:2454] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 0.
I20260926 13:03:58.643245 4224 leader_election.cc:302] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:03:58.643384 4647 raft_consensus.cc:2781] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:03:58.643435 4647 raft_consensus.cc:482] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:03:58.643458 4647 raft_consensus.cc:3037] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.644101 4647 raft_consensus.cc:504] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.644236 4647 leader_election.cc:288] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 election: Requested vote from peers 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:03:58.644378 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923f4440db24484e96d36e836f1ae78e" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75"
I20260926 13:03:58.644380 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923f4440db24484e96d36e836f1ae78e" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118"
I20260926 13:03:58.644448 4570 raft_consensus.cc:3037] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.644447 4430 raft_consensus.cc:3037] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:03:58.645215 4430 raft_consensus.cc:2454] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 1.
I20260926 13:03:58.645267 4570 raft_consensus.cc:2454] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 1.
I20260926 13:03:58.645412 4221 leader_election.cc:302] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c50ca5f20474abe8786996cd81118, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:03:58.645514 4647 raft_consensus.cc:2781] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:03:58.645654 4647 raft_consensus.cc:686] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 LEADER]: Becoming Leader. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Running, Role: LEADER
I20260926 13:03:58.645733 4647 consensus_queue.cc:237] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:03:58.646387 4131 catalog_manager.cc:5495] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "512b203f13cf4f3ba6e4330e91f7a621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260926 13:03:58.675662 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.673371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.2.86.1:44663)
0926 13:03:58.673413 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.673415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.673423 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:58.673498 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.673499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.673519 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.673601 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.673709 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.674170 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.674890 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.675093 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.675176 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.675323 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.675437 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.675522 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.675565 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.675584 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:03:58.680737 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.678361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:53371 (local address 127.2.86.3:38607)
0926 13:03:58.678433 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.678435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.678542 (+ 107us) server_negotiation.cc:407] Connection header received
0926 13:03:58.678610 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.678611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.678619 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.678670 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.678826 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.679269 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.680027 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.680222 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.680304 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.680484 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.680520 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.680575 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.680650 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.680668 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:03:58.680768 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.678270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.2.86.1:53371)
0926 13:03:58.678462 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.678471 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.678549 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.678673 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.678673 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.678674 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:58.678742 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.678743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.679369 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.679370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.679941 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.679942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.680324 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.680326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.680389 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.680402 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.680592 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.680594 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.680647 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.680685 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260926 13:03:58.682780 4430 raft_consensus.cc:1261] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Refusing update from remote peer 512b203f13cf4f3ba6e4330e91f7a621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.682842 4570 raft_consensus.cc:1261] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Refusing update from remote peer 512b203f13cf4f3ba6e4330e91f7a621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:03:58.683072 4647 consensus_queue.cc:1035] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.683195 4698 consensus_queue.cc:1035] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:03:58.690555 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.688410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:60631 (local address 127.2.86.1:44663)
0926 13:03:58.688526 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:03:58.688527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:03:58.688535 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:03:58.688554 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:03:58.688555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:03:58.688565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:03:58.688614 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:03:58.688708 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.689207 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.689855 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.690133 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.690195 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.690348 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.690380 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.690429 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:03:58.690487 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 13:03:58.690505 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 13:03:58.690567 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:03:58.688333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:44663 (local address 127.2.86.3:60631)
0926 13:03:58.688419 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 13:03:58.688428 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:03:58.688458 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:03:58.688618 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:03:58.688619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:03:58.688620 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:03:58.688656 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.688657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.689275 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.689276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.689778 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:03:58.689779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:03:58.690202 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:03:58.690203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:03:58.690258 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:03:58.690265 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:03:58.690440 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:03:58.690443 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:03:58.690483 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 13:03:58.690517 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 13:03:58.690886 4671 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 13:04:00.210662 4133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:00.210762 4133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:00.211552 4133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Sending DeleteTablet for 3 replicas of tablet 923f4440db24484e96d36e836f1ae78e
I20260926 13:04:00.211793 4410 tablet_service.cc:1514] Processing DeleteTablet for tablet 923f4440db24484e96d36e836f1ae78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51018
I20260926 13:04:00.211810 4550 tablet_service.cc:1514] Processing DeleteTablet for tablet 923f4440db24484e96d36e836f1ae78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51392
I20260926 13:04:00.211859 4270 tablet_service.cc:1514] Processing DeleteTablet for tablet 923f4440db24484e96d36e836f1ae78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:57770
I20260926 13:04:00.211982 4729 tablet_replica.cc:330] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: stopping tablet replica
I20260926 13:04:00.212040 4729 raft_consensus.cc:2229] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:00.212070 4729 raft_consensus.cc:2258] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.212339 4729 ts_tablet_manager.cc:1905] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.214429 4729 ts_tablet_manager.cc:1918] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:04:00.214496 4729 log.cc:1196] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/923f4440db24484e96d36e836f1ae78e
I20260926 13:04:00.214751 4729 ts_tablet_manager.cc:1939] T 923f4440db24484e96d36e836f1ae78e P 0dccbaffa54444a9989a2b34357bae75: Deleting consensus metadata
I20260926 13:04:00.214825 4730 tablet_replica.cc:330] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: stopping tablet replica
I20260926 13:04:00.214907 4730 raft_consensus.cc:2229] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:00.214953 4730 raft_consensus.cc:2258] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.215056 4731 tablet_replica.cc:330] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: stopping tablet replica
I20260926 13:04:00.215076 4120 catalog_manager.cc:4841] TS 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933): tablet 923f4440db24484e96d36e836f1ae78e (table test-table [id=632b44a92ee3437482c2955a1deca8f4]) successfully deleted
I20260926 13:04:00.215145 4731 raft_consensus.cc:2229] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:00.215234 4730 ts_tablet_manager.cc:1905] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.215263 4731 raft_consensus.cc:2258] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.215514 4731 ts_tablet_manager.cc:1905] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.216885 4730 ts_tablet_manager.cc:1918] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:04:00.216954 4730 log.cc:1196] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/923f4440db24484e96d36e836f1ae78e
I20260926 13:04:00.217166 4731 ts_tablet_manager.cc:1918] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:04:00.217216 4730 ts_tablet_manager.cc:1939] T 923f4440db24484e96d36e836f1ae78e P 2f9c50ca5f20474abe8786996cd81118: Deleting consensus metadata
I20260926 13:04:00.217229 4731 log.cc:1196] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/923f4440db24484e96d36e836f1ae78e
I20260926 13:04:00.217445 4731 ts_tablet_manager.cc:1939] T 923f4440db24484e96d36e836f1ae78e P 512b203f13cf4f3ba6e4330e91f7a621: Deleting consensus metadata
I20260926 13:04:00.217536 4117 catalog_manager.cc:4841] TS 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607): tablet 923f4440db24484e96d36e836f1ae78e (table test-table [id=632b44a92ee3437482c2955a1deca8f4]) successfully deleted
I20260926 13:04:00.217723 4120 catalog_manager.cc:4841] TS 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663): tablet 923f4440db24484e96d36e836f1ae78e (table test-table [id=632b44a92ee3437482c2955a1deca8f4]) successfully deleted
Sep 26 13:04:00 dist-test-slave-j4lq krb5kdc[4088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:04:00 dist-test-slave-j4lq krb5kdc[4088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:00.237031 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:00.229924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35278 (local address 127.2.86.62:38889)
0926 13:04:00.230057 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:00.230060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:00.230068 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:00.230129 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:00.230130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:00.230141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:00.230213 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:00.230757 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.231239 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.231956 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.232151 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.232222 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:00.234434 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:00.234436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:00.234436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:00.234443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:00.235799 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:00.236162 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:00.236163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:00.236163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:00.236225 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:00.236413 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:00.236414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:00.236414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:00.236516 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:00.236645 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:00.236764 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 13:04:00.236830 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260926 13:04:00.238402 4133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:00.238533 4133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:00.239817 4270 tablet_service.cc:1467] Processing CreateTablet for tablet c324084f48ec49d1a140e0d6f022f412 (DEFAULT_TABLE table=test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.239820 4410 tablet_service.cc:1467] Processing CreateTablet for tablet c324084f48ec49d1a140e0d6f022f412 (DEFAULT_TABLE table=test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.239820 4550 tablet_service.cc:1467] Processing CreateTablet for tablet c324084f48ec49d1a140e0d6f022f412 (DEFAULT_TABLE table=test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.239953 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c324084f48ec49d1a140e0d6f022f412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.239952 4550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c324084f48ec49d1a140e0d6f022f412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.239954 4270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c324084f48ec49d1a140e0d6f022f412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.240942 4740 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Bootstrap starting.
I20260926 13:04:00.241268 4740 tablet_bootstrap.cc:654] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.241314 4742 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Bootstrap starting.
I20260926 13:04:00.241720 4742 tablet_bootstrap.cc:654] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.241899 4740 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: No bootstrap required, opened a new log
I20260926 13:04:00.241956 4740 ts_tablet_manager.cc:1397] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:00.242003 4741 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Bootstrap starting.
I20260926 13:04:00.242061 4740 raft_consensus.cc:348] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.242130 4740 raft_consensus.cc:374] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.242178 4740 raft_consensus.cc:729] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.242242 4740 consensus_queue.cc:260] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.242311 4742 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: No bootstrap required, opened a new log
I20260926 13:04:00.242352 4740 ts_tablet_manager.cc:1428] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:00.242367 4742 ts_tablet_manager.cc:1397] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:00.242472 4742 raft_consensus.cc:348] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.242545 4742 raft_consensus.cc:374] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.242563 4742 raft_consensus.cc:729] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.242602 4742 consensus_queue.cc:260] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.242771 4742 ts_tablet_manager.cc:1428] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:00.243052 4741 tablet_bootstrap.cc:654] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.243790 4741 tablet_bootstrap.cc:492] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: No bootstrap required, opened a new log
I20260926 13:04:00.243872 4741 ts_tablet_manager.cc:1397] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:00.243978 4741 raft_consensus.cc:348] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.244062 4741 raft_consensus.cc:374] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.244095 4741 raft_consensus.cc:729] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.244148 4741 consensus_queue.cc:260] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.244236 4741 ts_tablet_manager.cc:1428] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:00.272564 4708 raft_consensus.cc:482] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:00.272645 4708 raft_consensus.cc:504] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.272802 4708 leader_election.cc:288] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:04:00.272967 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c324084f48ec49d1a140e0d6f022f412" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75" is_pre_election: true
I20260926 13:04:00.272981 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c324084f48ec49d1a140e0d6f022f412" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118" is_pre_election: true
I20260926 13:04:00.273064 4430 raft_consensus.cc:2454] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 0.
I20260926 13:04:00.273077 4570 raft_consensus.cc:2454] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 0.
I20260926 13:04:00.273231 4224 leader_election.cc:302] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:04:00.273317 4708 raft_consensus.cc:2781] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:00.273360 4708 raft_consensus.cc:482] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:00.273375 4708 raft_consensus.cc:3037] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.274098 4708 raft_consensus.cc:504] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.274227 4708 leader_election.cc:288] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 election: Requested vote from peers 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933), 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607)
I20260926 13:04:00.274361 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c324084f48ec49d1a140e0d6f022f412" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75"
I20260926 13:04:00.274401 4570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c324084f48ec49d1a140e0d6f022f412" candidate_uuid: "512b203f13cf4f3ba6e4330e91f7a621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9c50ca5f20474abe8786996cd81118"
I20260926 13:04:00.274441 4430 raft_consensus.cc:3037] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.274461 4570 raft_consensus.cc:3037] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.275193 4570 raft_consensus.cc:2454] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 1.
I20260926 13:04:00.275193 4430 raft_consensus.cc:2454] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512b203f13cf4f3ba6e4330e91f7a621 in term 1.
I20260926 13:04:00.275408 4224 leader_election.cc:302] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 2f9c50ca5f20474abe8786996cd81118, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:04:00.275508 4708 raft_consensus.cc:2781] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:00.275550 4708 raft_consensus.cc:686] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 LEADER]: Becoming Leader. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Running, Role: LEADER
I20260926 13:04:00.275593 4708 consensus_queue.cc:237] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.276186 4131 catalog_manager.cc:5495] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "512b203f13cf4f3ba6e4330e91f7a621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:00.291505 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:00.289066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.2.86.1:44663)
0926 13:04:00.289222 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:00.289231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:00.289238 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:00.289265 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:00.289266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:00.289275 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:00.289348 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:00.289509 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.290017 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.290795 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.291002 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.291082 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:00.291248 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:00.291279 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:00.291343 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:00.291396 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 13:04:00.291419 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260926 13:04:00.293574 4570 raft_consensus.cc:1261] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Refusing update from remote peer 512b203f13cf4f3ba6e4330e91f7a621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:00.293574 4430 raft_consensus.cc:1261] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Refusing update from remote peer 512b203f13cf4f3ba6e4330e91f7a621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:00.293753 4708 consensus_queue.cc:1035] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:00.293888 4721 consensus_queue.cc:1035] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:00.303447 4133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:00.303545 4133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:00.304246 4133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Sending DeleteTablet for 3 replicas of tablet c324084f48ec49d1a140e0d6f022f412
I20260926 13:04:00.304513 4410 tablet_service.cc:1514] Processing DeleteTablet for tablet c324084f48ec49d1a140e0d6f022f412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51018
I20260926 13:04:00.304520 4270 tablet_service.cc:1514] Processing DeleteTablet for tablet c324084f48ec49d1a140e0d6f022f412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:57770
I20260926 13:04:00.304605 4729 tablet_replica.cc:330] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: stopping tablet replica
I20260926 13:04:00.304659 4729 raft_consensus.cc:2229] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:00.304699 4729 raft_consensus.cc:2258] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.304713 4133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:00.304814 4133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:00.304811 4731 tablet_replica.cc:330] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: stopping tablet replica
I20260926 13:04:00.304863 4731 raft_consensus.cc:2229] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:00.304924 4731 raft_consensus.cc:2258] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.305054 4550 tablet_service.cc:1514] Processing DeleteTablet for tablet c324084f48ec49d1a140e0d6f022f412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:00 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51392
I20260926 13:04:00.305083 4729 ts_tablet_manager.cc:1905] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.305164 4730 tablet_replica.cc:330] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: stopping tablet replica
I20260926 13:04:00.305233 4730 raft_consensus.cc:2229] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:00.305291 4730 raft_consensus.cc:2258] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:00.305567 4731 ts_tablet_manager.cc:1905] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.305652 4730 ts_tablet_manager.cc:1905] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:00.306738 4729 ts_tablet_manager.cc:1918] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:00.306785 4729 log.cc:1196] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/c324084f48ec49d1a140e0d6f022f412
I20260926 13:04:00.306996 4729 ts_tablet_manager.cc:1939] T c324084f48ec49d1a140e0d6f022f412 P 0dccbaffa54444a9989a2b34357bae75: Deleting consensus metadata
I20260926 13:04:00.307063 4730 ts_tablet_manager.cc:1918] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:00.307111 4730 log.cc:1196] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/c324084f48ec49d1a140e0d6f022f412
I20260926 13:04:00.307157 4731 ts_tablet_manager.cc:1918] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:00.307206 4731 log.cc:1196] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/c324084f48ec49d1a140e0d6f022f412
I20260926 13:04:00.307350 4730 ts_tablet_manager.cc:1939] T c324084f48ec49d1a140e0d6f022f412 P 2f9c50ca5f20474abe8786996cd81118: Deleting consensus metadata
I20260926 13:04:00.307335 4270 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 (DEFAULT_TABLE table=test-table [id=e6a9ea3c1cb04bb699a08aff13647918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.307389 4410 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 (DEFAULT_TABLE table=test-table [id=e6a9ea3c1cb04bb699a08aff13647918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.307420 4731 ts_tablet_manager.cc:1939] T c324084f48ec49d1a140e0d6f022f412 P 512b203f13cf4f3ba6e4330e91f7a621: Deleting consensus metadata
I20260926 13:04:00.307451 4270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e2047ecc64f88be91b738b70ab054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.307477 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e2047ecc64f88be91b738b70ab054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.307653 4117 catalog_manager.cc:4841] TS 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607): tablet c324084f48ec49d1a140e0d6f022f412 (table test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]) successfully deleted
I20260926 13:04:00.307847 4550 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 (DEFAULT_TABLE table=test-table [id=e6a9ea3c1cb04bb699a08aff13647918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:00.307930 4550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7e2047ecc64f88be91b738b70ab054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:00.308349 4740 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: Bootstrap starting.
I20260926 13:04:00.308574 4742 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Bootstrap starting.
I20260926 13:04:00.308643 4740 tablet_bootstrap.cc:654] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.308892 4742 tablet_bootstrap.cc:654] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.309170 4120 catalog_manager.cc:4841] TS 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663): tablet c324084f48ec49d1a140e0d6f022f412 (table test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]) successfully deleted
I20260926 13:04:00.309334 4741 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Bootstrap starting.
I20260926 13:04:00.309399 4120 catalog_manager.cc:4841] TS 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933): tablet c324084f48ec49d1a140e0d6f022f412 (table test-table [id=c2ab79e0ab734de8b9f0f3f70a321c18]) successfully deleted
I20260926 13:04:00.309530 4742 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: No bootstrap required, opened a new log
I20260926 13:04:00.309569 4742 ts_tablet_manager.cc:1397] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:00.309654 4741 tablet_bootstrap.cc:654] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:00.309670 4742 raft_consensus.cc:348] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.309734 4742 raft_consensus.cc:374] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.309755 4740 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: No bootstrap required, opened a new log
I20260926 13:04:00.309758 4742 raft_consensus.cc:729] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dccbaffa54444a9989a2b34357bae75, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.309789 4740 ts_tablet_manager.cc:1397] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:00.309813 4742 consensus_queue.cc:260] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.309919 4742 ts_tablet_manager.cc:1428] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:00.309900 4740 raft_consensus.cc:348] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.309950 4740 raft_consensus.cc:374] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.309963 4740 raft_consensus.cc:729] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512b203f13cf4f3ba6e4330e91f7a621, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.310012 4740 consensus_queue.cc:260] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.310099 4740 ts_tablet_manager.cc:1428] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:00.310628 4741 tablet_bootstrap.cc:492] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: No bootstrap required, opened a new log
I20260926 13:04:00.310685 4741 ts_tablet_manager.cc:1397] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:00.310801 4741 raft_consensus.cc:348] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.310865 4741 raft_consensus.cc:374] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:00.310895 4741 raft_consensus.cc:729] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Initialized, Role: FOLLOWER
I20260926 13:04:00.310945 4741 consensus_queue.cc:260] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.311028 4741 ts_tablet_manager.cc:1428] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:00.535326 4716 raft_consensus.cc:482] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:00.535437 4716 raft_consensus.cc:504] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.535622 4716 leader_election.cc:288] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933)
I20260926 13:04:00.535863 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e2047ecc64f88be91b738b70ab054" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621" is_pre_election: true
I20260926 13:04:00.535948 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e2047ecc64f88be91b738b70ab054" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75" is_pre_election: true
I20260926 13:04:00.535985 4290 raft_consensus.cc:2454] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 0.
I20260926 13:04:00.536051 4430 raft_consensus.cc:2454] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 0.
I20260926 13:04:00.536163 4504 leader_election.cc:302] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9c50ca5f20474abe8786996cd81118, 512b203f13cf4f3ba6e4330e91f7a621; no voters:
I20260926 13:04:00.536279 4716 raft_consensus.cc:2781] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:00.536332 4716 raft_consensus.cc:482] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:00.536360 4716 raft_consensus.cc:3037] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.537177 4716 raft_consensus.cc:504] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.537304 4716 leader_election.cc:288] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 election: Requested vote from peers 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663), 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933)
I20260926 13:04:00.537451 4290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e2047ecc64f88be91b738b70ab054" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512b203f13cf4f3ba6e4330e91f7a621"
I20260926 13:04:00.537490 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7e2047ecc64f88be91b738b70ab054" candidate_uuid: "2f9c50ca5f20474abe8786996cd81118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dccbaffa54444a9989a2b34357bae75"
I20260926 13:04:00.537539 4290 raft_consensus.cc:3037] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.537547 4430 raft_consensus.cc:3037] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:00.538277 4430 raft_consensus.cc:2454] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 1.
I20260926 13:04:00.538277 4290 raft_consensus.cc:2454] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9c50ca5f20474abe8786996cd81118 in term 1.
I20260926 13:04:00.538467 4504 leader_election.cc:302] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dccbaffa54444a9989a2b34357bae75, 2f9c50ca5f20474abe8786996cd81118; no voters:
I20260926 13:04:00.538560 4716 raft_consensus.cc:2781] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:00.538609 4716 raft_consensus.cc:686] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9c50ca5f20474abe8786996cd81118, State: Running, Role: LEADER
I20260926 13:04:00.538678 4716 consensus_queue.cc:237] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } }
I20260926 13:04:00.539196 4131 catalog_manager.cc:5495] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "2f9c50ca5f20474abe8786996cd81118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9c50ca5f20474abe8786996cd81118" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260926 13:04:00.594883 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:00.592922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.2.86.3:38607)
0926 13:04:00.593064 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:00.593066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:00.593073 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:00.593094 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:00.593096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:00.593111 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:00.593160 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:00.593258 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.593703 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.594387 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.594560 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.594606 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:00.594702 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:00.594724 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:00.594754 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:00.594804 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 13:04:00.594818 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260926 13:04:00.598917 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:00.596997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38593 (local address 127.2.86.3:38607)
0926 13:04:00.597047 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:00.597048 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:00.597071 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:04:00.597139 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:00.597140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:00.597147 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:00.597191 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:00.597287 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.597702 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.598409 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.598583 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.598627 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:00.598729 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:00.598748 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:00.598783 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:00.598828 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 13:04:00.598852 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:00.598961 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:00.596880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38607 (local address 127.2.86.3:38593)
0926 13:04:00.597021 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:00.597031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:00.597087 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:00.597205 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:00.597206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:00.597207 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:04:00.597238 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:00.597239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.597756 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.597757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:00.598310 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:00.598331 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:00.598639 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:00.598640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:00.598674 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:00.598677 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:04:00.598792 (+ 115us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:04:00.598794 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:00.598825 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 13:04:00.598857 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260926 13:04:00.600692 4430 raft_consensus.cc:1261] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Refusing update from remote peer 2f9c50ca5f20474abe8786996cd81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:00.600682 4290 raft_consensus.cc:1261] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Refusing update from remote peer 2f9c50ca5f20474abe8786996cd81118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:00.600922 4716 consensus_queue.cc:1035] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "512b203f13cf4f3ba6e4330e91f7a621" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:00.601027 4711 consensus_queue.cc:1035] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dccbaffa54444a9989a2b34357bae75" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:04:00.605051 4709 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 13:04:02.121692 4133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:02.121806 4133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:02.122673 4133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0346ebecc9e0497d8cdc7958b3a26b7e: Sending DeleteTablet for 3 replicas of tablet 5c7e2047ecc64f88be91b738b70ab054
I20260926 13:04:02.122890 4270 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:02 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:57770
I20260926 13:04:02.122926 4410 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:02 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51018
I20260926 13:04:02.122941 4550 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c7e2047ecc64f88be91b738b70ab054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:02 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51392
I20260926 13:04:02.123121 4786 tablet_replica.cc:330] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: stopping tablet replica
I20260926 13:04:02.123131 4784 tablet_replica.cc:330] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621: stopping tablet replica
I20260926 13:04:02.123178 4784 raft_consensus.cc:2229] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:02.123212 4786 raft_consensus.cc:2229] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:02.123221 4784 raft_consensus.cc:2258] T 5c7e2047ecc64f88be91b738b70ab054 P 512b203f13cf4f3ba6e4330e91f7a621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:02.123270 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4205
I20260926 13:04:02.123962 4785 tablet_replica.cc:330] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: stopping tablet replica
I20260926 13:04:02.124017 4785 raft_consensus.cc:2229] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:02.124042 4785 raft_consensus.cc:2258] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:02.123330 4786 raft_consensus.cc:2258] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:02.124264 4786 ts_tablet_manager.cc:1905] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:02.124383 4785 ts_tablet_manager.cc:1905] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:02.125919 4785 ts_tablet_manager.cc:1918] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:04:02.125958 4785 log.cc:1196] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/5c7e2047ecc64f88be91b738b70ab054
I20260926 13:04:02.126116 4785 ts_tablet_manager.cc:1939] T 5c7e2047ecc64f88be91b738b70ab054 P 0dccbaffa54444a9989a2b34357bae75: Deleting consensus metadata
I20260926 13:04:02.126291 4786 ts_tablet_manager.cc:1918] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 13:04:02.126351 4786 log.cc:1196] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/5c7e2047ecc64f88be91b738b70ab054
I20260926 13:04:02.126374 4120 catalog_manager.cc:4841] TS 0dccbaffa54444a9989a2b34357bae75 (127.2.86.2:32933): tablet 5c7e2047ecc64f88be91b738b70ab054 (table test-table [id=e6a9ea3c1cb04bb699a08aff13647918]) successfully deleted
I20260926 13:04:02.126590 4786 ts_tablet_manager.cc:1939] T 5c7e2047ecc64f88be91b738b70ab054 P 2f9c50ca5f20474abe8786996cd81118: Deleting consensus metadata
I20260926 13:04:02.126866 4117 catalog_manager.cc:4841] TS 2f9c50ca5f20474abe8786996cd81118 (127.2.86.3:38607): tablet 5c7e2047ecc64f88be91b738b70ab054 (table test-table [id=e6a9ea3c1cb04bb699a08aff13647918]) successfully deleted
W20260926 13:04:02.130550 4120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:44663 (error 108)
I20260926 13:04:02.130720 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4345
I20260926 13:04:02.131593 4788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:02.131397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:44663 (local address 127.0.0.1:57642)
0926 13:04:02.131514 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:02.131563 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
W20260926 13:04:02.131693 4120 catalog_manager.cc:4584] TS 512b203f13cf4f3ba6e4330e91f7a621 (127.2.86.1:44663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c7e2047ecc64f88be91b738b70ab054: Network error: Client connection negotiation failed: client connection to 127.2.86.1:44663: connect: Connection refused (error 111)
I20260926 13:04:02.136406 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4485
I20260926 13:04:02.142275 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4101
2026-09-26T13:04:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6052 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:02 dist-test-slave-j4lq krb5kdc[4791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:02 dist-test-slave-j4lq krb5kdc[4791](info): set up 2 sockets
Sep 26 13:04:02 dist-test-slave-j4lq krb5kdc[4791](info): commencing operation
krb5kdc: starting...
W20260926 13:04:03.160673 2392 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:03.271469 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43327
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:43327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:03.348783 4804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:03.349030 4804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:03.349066 4804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:03.350207 4804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:03.350278 4804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:03.350313 4804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:03.350351 4804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:03.350381 4804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:03.351833 4804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:43327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:03.352102 4804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:03.352319 4804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:03.357604 4810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:03.357915 4812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.358426 4804 server_base.cc:1034] running on GCE node
W20260926 13:04:03.358474 4809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.358731 4804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:03.358978 4804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:03.360183 4804 hybrid_clock.cc:648] HybridClock initialized: now 1790427843360165 us; error 35 us; skew 500 ppm
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:03.368680 4804 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:03.369184 4804 webserver.cc:460] Webserver started at http://127.2.86.62:36089/ using document root <none> and password file <none>
I20260926 13:04:03.369413 4804 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:03.369481 4804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:03.369633 4804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:03.370689 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "a14a7212cc8e43e299bbca8fa27d6651"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "f1a9df419c650ad9c272555be867e894"
server_key_iv: "2bfa69091fbb58c8c9571514094db60d"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.371057 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "a14a7212cc8e43e299bbca8fa27d6651"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "f1a9df419c650ad9c272555be867e894"
server_key_iv: "2bfa69091fbb58c8c9571514094db60d"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.372351 4804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:03.373425 4819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.373615 4804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.373704 4804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "a14a7212cc8e43e299bbca8fa27d6651"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "f1a9df419c650ad9c272555be867e894"
server_key_iv: "2bfa69091fbb58c8c9571514094db60d"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.373759 4804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:03.385604 4804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:03.387276 4804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:03.387392 4804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:03.390888 4804 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:43327
I20260926 13:04:03.390923 4871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:43327 every 8 connection(s)
I20260926 13:04:03.391252 4804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:03.392056 4872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:03.394220 4872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651: Bootstrap starting.
I20260926 13:04:03.394743 4872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:03.395061 4872 log.cc:824] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:03.395681 4872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651: No bootstrap required, opened a new log
I20260926 13:04:03.395692 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4804
I20260926 13:04:03.395792 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:03.395982 2392 external_mini_cluster.cc:1428] Setting key db83f56bb64f20f3e8587f71c24dc2be
I20260926 13:04:03.397653 4872 raft_consensus.cc:348] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } }
I20260926 13:04:03.397799 4872 raft_consensus.cc:374] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:03.397852 4872 raft_consensus.cc:729] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14a7212cc8e43e299bbca8fa27d6651, State: Initialized, Role: FOLLOWER
I20260926 13:04:03.398028 4872 consensus_queue.cc:260] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } }
I20260926 13:04:03.398113 4872 raft_consensus.cc:388] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:03.398168 4872 raft_consensus.cc:482] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:03.398241 4872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:03.398744 4872 raft_consensus.cc:504] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } }
I20260926 13:04:03.398854 4872 leader_election.cc:302] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a14a7212cc8e43e299bbca8fa27d6651; no voters:
I20260926 13:04:03.399027 4872 leader_election.cc:288] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:03.399055 4877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:03.399174 4877 raft_consensus.cc:686] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [term 1 LEADER]: Becoming Leader. State: Replica: a14a7212cc8e43e299bbca8fa27d6651, State: Running, Role: LEADER
I20260926 13:04:03.399263 4872 sys_catalog.cc:564] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:03.399248 4877 consensus_queue.cc:237] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } }
I20260926 13:04:03.399726 4878 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a14a7212cc8e43e299bbca8fa27d6651. Latest consensus state: current_term: 1 leader_uuid: "a14a7212cc8e43e299bbca8fa27d6651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } } }
I20260926 13:04:03.399847 4878 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:03.400017 4879 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a14a7212cc8e43e299bbca8fa27d6651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14a7212cc8e43e299bbca8fa27d6651" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43327 } } }
I20260926 13:04:03.400099 4879 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:03.400312 4886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:03.400787 4886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:03.402606 4886 catalog_manager.cc:1347] Generated new cluster ID: 7e73c9089c41429f8237510d6540fe7a
I20260926 13:04:03.402652 4886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:03.408380 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.396442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.2.86.62:43327)
0926 13:04:03.396624 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:03.396633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:03.396662 (+ 29us) server_negotiation.cc:407] Connection header received
0926 13:04:03.397883 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:03.397892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:03.397967 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:03.398125 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:03.398832 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.400170 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.401366 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.401882 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.402010 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.404665 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:03.404669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:03.404669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:03.404680 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:03.406819 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.407219 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.407222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.407222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.407325 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.407519 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.407520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.407520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.407810 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:03.407984 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:03.408052 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 13:04:03.408185 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260926 13:04:03.412586 4886 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:03.413162 4886 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:03.416635 4886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a14a7212cc8e43e299bbca8fa27d6651: Generated new TSK 0
I20260926 13:04:03.416815 4886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:03.445731 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:03.524434 4898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:03.524655 4898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:03.524695 4898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:03.525916 4898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:03.525981 4898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:03.526026 4898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:03.527412 4898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:03.527666 4898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:03.527882 4898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:03.532624 4906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.532716 4898 server_base.cc:1034] running on GCE node
W20260926 13:04:03.532605 4903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:03.532629 4904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.533164 4898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:03.533408 4898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:03.534574 4898 hybrid_clock.cc:648] HybridClock initialized: now 1790427843534570 us; error 36 us; skew 500 ppm
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:03.537041 4898 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:03.537446 4898 webserver.cc:460] Webserver started at http://127.2.86.1:41037/ using document root <none> and password file <none>
I20260926 13:04:03.537664 4898 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:03.537729 4898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:03.537885 4898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:03.538857 4898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "550f36df48d74c33b5e10d74c5773e58"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "d0c73b91c18f402eb4b60b8c4d31d48b"
server_key_iv: "5150b7e3506fd6c939d2de4593c51108"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.539201 4898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "550f36df48d74c33b5e10d74c5773e58"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "d0c73b91c18f402eb4b60b8c4d31d48b"
server_key_iv: "5150b7e3506fd6c939d2de4593c51108"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.540534 4898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:03.541435 4913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.541599 4898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:04:03.541712 4898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "550f36df48d74c33b5e10d74c5773e58"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "d0c73b91c18f402eb4b60b8c4d31d48b"
server_key_iv: "5150b7e3506fd6c939d2de4593c51108"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.541790 4898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:03.559139 4898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:03.561396 4898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:03.561549 4898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:03.561766 4898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:03.562058 4898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:03.562112 4898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.562153 4898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:03.562196 4898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.567993 4898 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:43339
I20260926 13:04:03.568073 5026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:43339 every 8 connection(s)
I20260926 13:04:03.568377 4898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:03.570043 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 4898
I20260926 13:04:03.570148 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:03.570370 2392 external_mini_cluster.cc:1428] Setting key faed11bbeba56a049e9c21a6671bfea1
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
I20260926 13:04:03.579191 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.569106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:60553 (local address 127.2.86.62:43327)
0926 13:04:03.569187 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:03.569189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:03.569345 (+ 156us) server_negotiation.cc:407] Connection header received
0926 13:04:03.569862 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:03.569863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:03.569874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:03.569918 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:03.571138 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.571630 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.572968 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.573300 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.573377 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.575928 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:03.575929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:03.575930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:03.575936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:03.577738 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.578230 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.578231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.578232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.578293 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.578534 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.578535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.578536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.578676 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:03.578745 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:03.579009 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 13:04:03.579100 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:04:03.579401 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.569060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43327 (local address 127.2.86.1:60553)
0926 13:04:03.569223 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:03.569257 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:03.569765 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:03.569994 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:03.569999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:03.570753 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:03.571024 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.571029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.571721 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.571722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.572830 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.572833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.573390 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.573390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.573494 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.574028 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:03.574036 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:03.575835 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:03.577824 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.577832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.577834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.578170 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.578373 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.578373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.578373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.578469 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.578739 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:03.578743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:03.578854 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 13:04:03.578973 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:04:03.579129 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260926 13:04:03.579909 5027 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43327
I20260926 13:04:03.580009 5027 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:03.580195 5027 heartbeater.cc:511] Master 127.2.86.62:43327 requested a full tablet report, sending...
I20260926 13:04:03.580783 4836 ts_manager.cc:194] Registered new tserver with Master: 550f36df48d74c33b5e10d74c5773e58 (127.2.86.1:43339)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
I20260926 13:04:03.581674 4836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:60553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:03.596087 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:03.675603 5032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:03.675792 5032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:03.675814 5032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:03.677006 5032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:03.677044 5032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:03.677096 5032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:03.678499 5032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:03.678675 5032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:03.678846 5032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:03.683586 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:03.683574 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.683682 5032 server_base.cc:1034] running on GCE node
W20260926 13:04:03.683575 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.684007 5032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:03.684237 5032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:03.685377 5032 hybrid_clock.cc:648] HybridClock initialized: now 1790427843685356 us; error 37 us; skew 500 ppm
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:03.687737 5032 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:03.688172 5032 webserver.cc:460] Webserver started at http://127.2.86.2:44727/ using document root <none> and password file <none>
I20260926 13:04:03.688411 5032 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:03.688454 5032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:03.688609 5032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:03.689687 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "e1f6cc83161143d79bced48c9c4c6399"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "3417906e2ca69d432e907531973c02cb"
server_key_iv: "c0b02b1fb018d97b3e1018d20c4f8841"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.690042 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "e1f6cc83161143d79bced48c9c4c6399"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "3417906e2ca69d432e907531973c02cb"
server_key_iv: "c0b02b1fb018d97b3e1018d20c4f8841"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.691296 5032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:04:03.691982 5047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.692170 5032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:03.692250 5032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "e1f6cc83161143d79bced48c9c4c6399"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "3417906e2ca69d432e907531973c02cb"
server_key_iv: "c0b02b1fb018d97b3e1018d20c4f8841"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.692301 5032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:03.712224 5032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:03.714556 5032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:03.714670 5032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:03.714851 5032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:03.715178 5032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:03.715214 5032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.715235 5032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:03.715284 5032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.720705 5032 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:46381
I20260926 13:04:03.720767 5160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:46381 every 8 connection(s)
I20260926 13:04:03.721098 5032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:03.729841 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.721827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:38609 (local address 127.2.86.62:43327)
0926 13:04:03.721890 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:03.721892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:03.722091 (+ 199us) server_negotiation.cc:407] Connection header received
0926 13:04:03.722538 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:03.722539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:03.722549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:03.722598 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:03.723480 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.723959 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.724860 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.725048 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.725096 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.727003 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:03.727004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:03.727005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:03.727012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:03.728474 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.728894 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.728895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.728896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.728955 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.729200 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.729201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.729201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.729317 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:03.729410 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:03.729673 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 13:04:03.729749 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:03.730042 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.721806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43327 (local address 127.2.86.2:38609)
0926 13:04:03.721986 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:03.722020 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:03.722462 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:03.722653 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:03.722656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:03.723223 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:03.723408 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.723412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.724034 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.724034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.724794 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.724795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.725116 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.725116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.725210 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.725620 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:03.725628 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:03.726906 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:03.728553 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.728560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.728562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.728779 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.729021 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.729022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.729023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.729131 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.729405 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:03.729411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:03.729544 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 13:04:03.729651 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 13:04:03.729803 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260926 13:04:03.730494 5161 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43327
I20260926 13:04:03.730589 5161 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:03.730793 5161 heartbeater.cc:511] Master 127.2.86.62:43327 requested a full tablet report, sending...
I20260926 13:04:03.730791 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5032
I20260926 13:04:03.730870 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:03.731094 2392 external_mini_cluster.cc:1428] Setting key 1e3dba44068cb76904ba5f1bbd1628e1
I20260926 13:04:03.731211 4836 ts_manager.cc:194] Registered new tserver with Master: e1f6cc83161143d79bced48c9c4c6399 (127.2.86.2:46381)
I20260926 13:04:03.731855 4836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:38609
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:03.753261 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:03.830672 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:03.830876 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:03.830897 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:03.832019 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:03.832059 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:03.832110 5166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:03.833508 5166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:03.833729 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:03.833940 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:03.838615 5172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:03.838622 5174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.838654 5166 server_base.cc:1034] running on GCE node
W20260926 13:04:03.838620 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:03.838991 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:03.839226 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:03.840359 5166 hybrid_clock.cc:648] HybridClock initialized: now 1790427843840340 us; error 33 us; skew 500 ppm
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:03.842658 5166 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:03.843113 5166 webserver.cc:460] Webserver started at http://127.2.86.3:37761/ using document root <none> and password file <none>
I20260926 13:04:03.843317 5166 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:03.843359 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:03.843505 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:03.844431 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "465a6b4ed90947fd940cb0c1485ef923"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "7c7f6c7e7e383dbb84c29a51c5621dad"
server_key_iv: "7e057f3043ee73bb68a4be3b288015c8"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.844801 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "465a6b4ed90947fd940cb0c1485ef923"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "7c7f6c7e7e383dbb84c29a51c5621dad"
server_key_iv: "7e057f3043ee73bb68a4be3b288015c8"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.846093 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:03.846830 5181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.847028 5166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:03.847116 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "465a6b4ed90947fd940cb0c1485ef923"
format_stamp: "Formatted at 2026-09-26 13:04:03 on dist-test-slave-j4lq"
server_key: "7c7f6c7e7e383dbb84c29a51c5621dad"
server_key_iv: "7e057f3043ee73bb68a4be3b288015c8"
server_key_version: "encryptionkey@0"
I20260926 13:04:03.847191 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:03.867283 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:03.869289 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:03.869457 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:03.869680 5166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:03.869999 5166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:03.870050 5166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.870088 5166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:03.870115 5166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:03.876436 5166 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38205
I20260926 13:04:03.876506 5294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38205 every 8 connection(s)
I20260926 13:04:03.876904 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:03.877413 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5166
I20260926 13:04:03.877513 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:03.877740 2392 external_mini_cluster.cc:1428] Setting key 5655465454121791aee8b07bef483787
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[4791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:03.887821 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.878358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:55753 (local address 127.2.86.62:43327)
0926 13:04:03.878437 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:03.878440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:03.878881 (+ 441us) server_negotiation.cc:407] Connection header received
0926 13:04:03.879502 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:03.879503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:03.879512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:03.879545 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:03.880669 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.881389 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.882631 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.882916 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.882986 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.885104 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:03.885106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:03.885106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:03.885113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:03.886514 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.886933 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.886935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.886935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.886991 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:03.887225 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:03.887226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:03.887226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:03.887343 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:03.887414 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:03.887662 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 13:04:03.887734 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:03.888010 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:03.878312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43327 (local address 127.2.86.3:55753)
0926 13:04:03.878728 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:03.878769 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:03.879409 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:03.879614 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:03.879618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:03.880306 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:03.880580 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.880587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.881475 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.881476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.882541 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:03.882543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:03.883015 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:03.883016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:03.883154 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:03.883671 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:03.883678 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:03.885011 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:03.886615 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.886622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.886624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.886849 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.887054 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:03.887055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:03.887055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:03.887162 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:03.887406 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:03.887412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:03.887541 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 13:04:03.887642 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 13:04:03.887791 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":73,"threads_started":1}
I20260926 13:04:03.888412 5295 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43327
I20260926 13:04:03.888512 5295 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:03.888671 5295 heartbeater.cc:511] Master 127.2.86.62:43327 requested a full tablet report, sending...
I20260926 13:04:03.889097 4836 ts_manager.cc:194] Registered new tserver with Master: 465a6b4ed90947fd940cb0c1485ef923 (127.2.86.3:38205)
I20260926 13:04:03.889427 4836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:55753
I20260926 13:04:03.890033 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:04:03.900815 4876 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:03.899915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.2.86.62:43327)
0926 13:04:03.899965 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:03.899967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:03.900093 (+ 126us) server_negotiation.cc:407] Connection header received
0926 13:04:03.900188 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:03.900189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:03.900199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:03.900270 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:03.900753 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41646: BlockingRecv error: recv got EOF from 127.0.0.1:41646 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 13:04:03.901655 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4898
I20260926 13:04:03.905788 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5032
I20260926 13:04:03.909978 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5166
I20260926 13:04:03.913769 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 4804
2026-09-26T13:04:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1768 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[5307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[5307](info): set up 2 sockets
Sep 26 13:04:03 dist-test-slave-j4lq krb5kdc[5307](info): commencing operation
krb5kdc: starting...
W20260926 13:04:04.932570 2392 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:05.043643 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39101
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:39101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:05.119886 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:05.120070 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:05.120091 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:05.121126 5320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:05.121157 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:05.121171 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:05.121184 5320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:05.121196 5320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:05.122563 5320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:39101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:05.122756 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:05.122912 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:05.127537 5325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:05.127583 5326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:05.127622 5320 server_base.cc:1034] running on GCE node
W20260926 13:04:05.127602 5328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:05.128008 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:05.128315 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:05.129465 5320 hybrid_clock.cc:648] HybridClock initialized: now 1790427845129472 us; error 46 us; skew 500 ppm
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427845, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:05.131707 5320 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:05.132141 5320 webserver.cc:460] Webserver started at http://127.2.86.62:45929/ using document root <none> and password file <none>
I20260926 13:04:05.132360 5320 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:05.132427 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:05.132553 5320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:05.133651 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "e063dc365d4a48eabde26d011b05882e"
format_stamp: "Formatted at 2026-09-26 13:04:05 on dist-test-slave-j4lq"
server_key: "95f5c837c5fb42f79d4f9c6a59e4d346"
server_key_iv: "133cabb4af2daeb1f82f958f0c6b6f9e"
server_key_version: "encryptionkey@0"
I20260926 13:04:05.134013 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "e063dc365d4a48eabde26d011b05882e"
format_stamp: "Formatted at 2026-09-26 13:04:05 on dist-test-slave-j4lq"
server_key: "95f5c837c5fb42f79d4f9c6a59e4d346"
server_key_iv: "133cabb4af2daeb1f82f958f0c6b6f9e"
server_key_version: "encryptionkey@0"
I20260926 13:04:05.135280 5320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:05.135973 5335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:05.136133 5320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:05.136228 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "e063dc365d4a48eabde26d011b05882e"
format_stamp: "Formatted at 2026-09-26 13:04:05 on dist-test-slave-j4lq"
server_key: "95f5c837c5fb42f79d4f9c6a59e4d346"
server_key_iv: "133cabb4af2daeb1f82f958f0c6b6f9e"
server_key_version: "encryptionkey@0"
I20260926 13:04:05.136303 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:05.155270 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:05.157140 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:05.157290 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:05.161152 5320 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:39101
I20260926 13:04:05.161186 5387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:39101 every 8 connection(s)
I20260926 13:04:05.161535 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:05.162369 5388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:05.164667 5388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e: Bootstrap starting.
I20260926 13:04:05.165189 5388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:05.165429 5388 log.cc:824] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:05.166015 5388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e: No bootstrap required, opened a new log
I20260926 13:04:05.167475 5388 raft_consensus.cc:348] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } }
I20260926 13:04:05.167620 5388 raft_consensus.cc:374] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:05.167644 5388 raft_consensus.cc:729] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e063dc365d4a48eabde26d011b05882e, State: Initialized, Role: FOLLOWER
I20260926 13:04:05.167800 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5320
I20260926 13:04:05.167804 5388 consensus_queue.cc:260] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } }
I20260926 13:04:05.167889 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:05.167893 5388 raft_consensus.cc:388] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:05.167924 5388 raft_consensus.cc:482] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:05.167972 5388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:05.168087 2392 external_mini_cluster.cc:1428] Setting key bfdfe21defd168ddb765b64073cef96c
I20260926 13:04:05.169044 5388 raft_consensus.cc:504] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } }
I20260926 13:04:05.169158 5388 leader_election.cc:302] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e063dc365d4a48eabde26d011b05882e; no voters:
I20260926 13:04:05.169353 5393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:05.169328 5388 leader_election.cc:288] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:05.169513 5393 raft_consensus.cc:686] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [term 1 LEADER]: Becoming Leader. State: Replica: e063dc365d4a48eabde26d011b05882e, State: Running, Role: LEADER
I20260926 13:04:05.169668 5388 sys_catalog.cc:564] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:05.169634 5393 consensus_queue.cc:237] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } }
I20260926 13:04:05.169986 5393 sys_catalog.cc:455] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e063dc365d4a48eabde26d011b05882e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } } }
I20260926 13:04:05.170106 5393 sys_catalog.cc:458] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:05.170133 5394 sys_catalog.cc:455] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e063dc365d4a48eabde26d011b05882e. Latest consensus state: current_term: 1 leader_uuid: "e063dc365d4a48eabde26d011b05882e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e063dc365d4a48eabde26d011b05882e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39101 } } }
I20260926 13:04:05.170214 5394 sys_catalog.cc:458] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:05.170852 5400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:05.171442 5400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:05.173202 5400 catalog_manager.cc:1347] Generated new cluster ID: 82b0d84602a74ebda516875bc519607a
I20260926 13:04:05.173245 5400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:05.178607 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:05.168544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.2.86.62:39101)
0926 13:04:05.168742 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:05.168749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:05.168772 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:04:05.168860 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:05.168907 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:05.168961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:05.169065 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:05.169890 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:05.171435 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:05.172555 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:05.173086 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:05.173234 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:05.175380 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:05.175384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:05.175384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:05.175395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:05.177117 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:05.177506 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:05.177509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:05.177509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:05.177607 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:05.177812 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:05.177813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:05.177813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:05.178038 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:05.178192 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:05.178283 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 13:04:05.178417 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260926 13:04:05.179502 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:04:05.184924 5400 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:05.185479 5400 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:05.192926 5400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e063dc365d4a48eabde26d011b05882e: Generated new TSK 0
I20260926 13:04:05.193084 5400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:05.197059 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:05.193192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.2.86.62:39101)
0926 13:04:05.193229 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:05.193231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:05.193467 (+ 236us) server_negotiation.cc:407] Connection header received
0926 13:04:05.193517 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:05.193518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:05.193526 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:05.193567 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:05.193957 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:05.194391 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:05.195447 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:05.195933 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:05.195981 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:05.196579 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:05.196580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:05.196580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:05.196752 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:05.196765 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:05.196770 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:05.196814 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:05.196945 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 13:04:05.196965 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:04:05.199316 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5320
2026-09-26T13:04:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1283 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5421](info): set up 2 sockets
Sep 26 13:04:05 dist-test-slave-j4lq krb5kdc[5421](info): commencing operation
krb5kdc: starting...
W20260926 13:04:06.220636 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:06.329598 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:36441
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:36441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:06.408077 5434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:06.408286 5434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:06.408319 5434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:06.409528 5434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:06.409588 5434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:06.409612 5434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:06.409633 5434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:06.409662 5434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:06.411072 5434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:36441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:36441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:06.411330 5434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:06.411546 5434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:06.416160 5439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.416169 5440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.416208 5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:06.416425 5434 server_base.cc:1034] running on GCE node
I20260926 13:04:06.416659 5434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:06.417039 5434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:06.418195 5434 hybrid_clock.cc:648] HybridClock initialized: now 1790427846418169 us; error 42 us; skew 500 ppm
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:06.420540 5434 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:06.421062 5434 webserver.cc:460] Webserver started at http://127.2.86.62:38077/ using document root <none> and password file <none>
I20260926 13:04:06.421286 5434 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:06.421351 5434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:06.421483 5434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:06.422492 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "5c27b267b833433cae6385c65e1417a7"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "cebb04dc2fc48bc15817e4eda41cac21"
server_key_iv: "9a6284603ddaf44fa0ff6845465ba7b8"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.422863 5434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "5c27b267b833433cae6385c65e1417a7"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "cebb04dc2fc48bc15817e4eda41cac21"
server_key_iv: "9a6284603ddaf44fa0ff6845465ba7b8"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.424150 5434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:06.424799 5449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.424973 5434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:06.425076 5434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "5c27b267b833433cae6385c65e1417a7"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "cebb04dc2fc48bc15817e4eda41cac21"
server_key_iv: "9a6284603ddaf44fa0ff6845465ba7b8"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.425146 5434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:06.434224 5434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:06.435883 5434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:06.436036 5434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:06.439836 5434 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:36441
I20260926 13:04:06.439865 5501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:36441 every 8 connection(s)
I20260926 13:04:06.440196 5434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:06.441162 5502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:06.443347 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5434
I20260926 13:04:06.443452 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:06.443634 2392 external_mini_cluster.cc:1428] Setting key e4912ef605eea1eb723dcec78e36860b
I20260926 13:04:06.443634 5502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7: Bootstrap starting.
I20260926 13:04:06.444340 5502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:06.444630 5502 log.cc:824] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:06.445412 5502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7: No bootstrap required, opened a new log
I20260926 13:04:06.447556 5502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } }
I20260926 13:04:06.447741 5502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:06.447795 5502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c27b267b833433cae6385c65e1417a7, State: Initialized, Role: FOLLOWER
I20260926 13:04:06.447968 5502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } }
I20260926 13:04:06.448062 5502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:06.448091 5502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:06.448124 5502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:06.448776 5502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } }
I20260926 13:04:06.448921 5502 leader_election.cc:302] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c27b267b833433cae6385c65e1417a7; no voters:
I20260926 13:04:06.449092 5502 leader_election.cc:288] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:06.449121 5507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:06.449266 5507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5c27b267b833433cae6385c65e1417a7, State: Running, Role: LEADER
I20260926 13:04:06.449363 5502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:06.449425 5507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } }
I20260926 13:04:06.449883 5507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c27b267b833433cae6385c65e1417a7. Latest consensus state: current_term: 1 leader_uuid: "5c27b267b833433cae6385c65e1417a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } } }
I20260926 13:04:06.449904 5508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c27b267b833433cae6385c65e1417a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c27b267b833433cae6385c65e1417a7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36441 } } }
I20260926 13:04:06.449981 5507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:06.450021 5508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:06.450290 5516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:06.450778 5516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:06.452780 5516 catalog_manager.cc:1347] Generated new cluster ID: 5eb891c6afbc49579071897a2ff909be
I20260926 13:04:06.452842 5516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:06.454928 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.444143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.2.86.62:36441)
0926 13:04:06.444306 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:06.444315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:06.444341 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:04:06.444965 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:06.444974 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:06.445037 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:06.445173 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:06.445945 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.446770 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.447900 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.448512 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.448626 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.451563 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:06.451567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:06.451568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:06.451578 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:06.453463 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.453829 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.453832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.453833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.453917 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.454120 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.454121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.454121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.454348 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:06.454517 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:06.454594 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:04:06.454723 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260926 13:04:06.464259 5516 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:06.464758 5516 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:06.475553 5516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c27b267b833433cae6385c65e1417a7: Generated new TSK 0
I20260926 13:04:06.475739 5516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:06.503479 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:06.601091 5528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:06.601279 5528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:06.601302 5528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:06.602428 5528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:06.602468 5528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:06.602492 5528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:06.603868 5528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:06.604104 5528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:06.604341 5528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:06.609114 5536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.609141 5534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:06.609274 5528 server_base.cc:1034] running on GCE node
W20260926 13:04:06.609143 5533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:06.609555 5528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:06.609797 5528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:06.610952 5528 hybrid_clock.cc:648] HybridClock initialized: now 1790427846610935 us; error 34 us; skew 500 ppm
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:06.613411 5528 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:06.613826 5528 webserver.cc:460] Webserver started at http://127.2.86.1:35231/ using document root <none> and password file <none>
I20260926 13:04:06.613999 5528 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:06.614038 5528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:06.614136 5528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:06.615140 5528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "3434860efeb84eb8b99dd0bceabee049"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "a5c7c2e39614cbae98f5c7bc7e95df7b"
server_key_iv: "aa8a386cc241885f1e437752e382faa4"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.615485 5528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "3434860efeb84eb8b99dd0bceabee049"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "a5c7c2e39614cbae98f5c7bc7e95df7b"
server_key_iv: "aa8a386cc241885f1e437752e382faa4"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.616678 5528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:06.617409 5543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.617554 5528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.617631 5528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "3434860efeb84eb8b99dd0bceabee049"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "a5c7c2e39614cbae98f5c7bc7e95df7b"
server_key_iv: "aa8a386cc241885f1e437752e382faa4"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.617681 5528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:06.629889 5528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:06.632097 5528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:06.632246 5528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:06.632469 5528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:06.632757 5528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:06.632786 5528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.632838 5528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:06.632886 5528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.638420 5528 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:32873
I20260926 13:04:06.638487 5656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:32873 every 8 connection(s)
I20260926 13:04:06.638796 5528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:06.647040 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.639476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:39919 (local address 127.2.86.62:36441)
0926 13:04:06.639539 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:06.639541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:06.639731 (+ 190us) server_negotiation.cc:407] Connection header received
0926 13:04:06.640155 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:06.640156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:06.640165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:06.640217 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:06.641089 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.641557 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.642492 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.642692 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.642739 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.644553 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:06.644555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:06.644555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:06.644561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:06.645993 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.646313 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.646313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.646314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.646361 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.646553 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.646553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.646554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.646645 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:06.646691 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:06.646907 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 13:04:06.646965 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:04:06.647202 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.639451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:36441 (local address 127.2.86.1:39919)
0926 13:04:06.639614 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:06.639652 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:06.640081 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:06.640264 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:06.640267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:06.640773 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:06.641014 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.641019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.641649 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.641649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.642420 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.642422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.642756 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.642757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.642843 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.643204 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:06.643211 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:06.644473 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:06.646057 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.646061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.646063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.646252 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.646415 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.646416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.646416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.646501 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.646696 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:06.646700 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:06.646797 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 13:04:06.646882 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 13:04:06.646987 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260926 13:04:06.647652 5657 heartbeater.cc:347] Connected to a master server at 127.2.86.62:36441
I20260926 13:04:06.647722 5657 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:06.647841 5657 heartbeater.cc:511] Master 127.2.86.62:36441 requested a full tablet report, sending...
I20260926 13:04:06.648269 5466 ts_manager.cc:194] Registered new tserver with Master: 3434860efeb84eb8b99dd0bceabee049 (127.2.86.1:32873)
I20260926 13:04:06.648301 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5528
I20260926 13:04:06.648392 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:06.648587 2392 external_mini_cluster.cc:1428] Setting key 8fede8c9bc3ee184b2dfed9654bff551
I20260926 13:04:06.649183 5466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:39919
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:06.671329 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:06.750360 5662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:06.750564 5662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:06.750598 5662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:06.751691 5662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:06.751751 5662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:06.751794 5662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:06.753305 5662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:06.753561 5662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:06.753780 5662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:06.758594 5667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.758589 5668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.758606 5670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:06.758759 5662 server_base.cc:1034] running on GCE node
I20260926 13:04:06.758983 5662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:06.759227 5662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:06.760368 5662 hybrid_clock.cc:648] HybridClock initialized: now 1790427846760359 us; error 27 us; skew 500 ppm
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:06.762837 5662 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:06.763338 5662 webserver.cc:460] Webserver started at http://127.2.86.2:34197/ using document root <none> and password file <none>
I20260926 13:04:06.763559 5662 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:06.763602 5662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:06.763744 5662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:06.764712 5662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "cac7f9ca00c54644b5798b5ea66117c5"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "030bc2703d45d7571f876e3840b47606"
server_key_iv: "4f3fcdb13cda0a814c5723749e8d3c76"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.765141 5662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "cac7f9ca00c54644b5798b5ea66117c5"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "030bc2703d45d7571f876e3840b47606"
server_key_iv: "4f3fcdb13cda0a814c5723749e8d3c76"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.766530 5662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:06.767225 5677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.767382 5662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.767458 5662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "cac7f9ca00c54644b5798b5ea66117c5"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "030bc2703d45d7571f876e3840b47606"
server_key_iv: "4f3fcdb13cda0a814c5723749e8d3c76"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.767509 5662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:06.774008 5662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:06.776113 5662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:06.776214 5662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:06.776422 5662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:06.776648 5662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:06.776678 5662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.776700 5662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:06.776712 5662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.782126 5662 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:36159
I20260926 13:04:06.782189 5790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:36159 every 8 connection(s)
I20260926 13:04:06.782508 5662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:06.785349 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5662
I20260926 13:04:06.785441 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:06.785655 2392 external_mini_cluster.cc:1428] Setting key 2921e85a176ffd7d35ad44126a9e5c2c
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:06.792151 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.783207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:48393 (local address 127.2.86.62:36441)
0926 13:04:06.783269 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:06.783271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:06.783432 (+ 161us) server_negotiation.cc:407] Connection header received
0926 13:04:06.783922 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:06.783923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:06.783933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:06.783984 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:06.784811 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.785512 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.786670 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.787000 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.787072 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.789236 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:06.789237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:06.789238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:06.789246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:06.790890 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.791248 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.791249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.791249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.791311 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.791538 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.791539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.791540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.791676 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:06.791747 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:06.791977 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 13:04:06.792056 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:04:06.792183 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.783192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:36441 (local address 127.2.86.2:48393)
0926 13:04:06.783336 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:06.783370 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:06.783835 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:06.784044 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:06.784047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:06.784568 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:06.784742 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.784746 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.785597 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.785597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.786593 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.786596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.787078 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.787079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.787172 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.787711 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:06.787719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:06.789151 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:06.790969 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.790974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.790976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.791174 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.791379 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.791379 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.791380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.791475 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.791741 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:06.791744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:06.791853 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 13:04:06.791930 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:04:06.792036 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260926 13:04:06.792686 5791 heartbeater.cc:347] Connected to a master server at 127.2.86.62:36441
I20260926 13:04:06.792790 5791 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:06.793004 5791 heartbeater.cc:511] Master 127.2.86.62:36441 requested a full tablet report, sending...
I20260926 13:04:06.793417 5466 ts_manager.cc:194] Registered new tserver with Master: cac7f9ca00c54644b5798b5ea66117c5 (127.2.86.2:36159)
I20260926 13:04:06.794016 5466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:48393
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:06.811549 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:06.889160 5796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:06.889415 5796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:06.889438 5796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:06.890483 5796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:06.890515 5796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:06.890539 5796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:06.891812 5796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:36441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:06.891978 5796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:06.892125 5796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:06.896695 5804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.896703 5802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:06.896754 5801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:06.896951 5796 server_base.cc:1034] running on GCE node
I20260926 13:04:06.897131 5796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:06.897308 5796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:06.898483 5796 hybrid_clock.cc:648] HybridClock initialized: now 1790427846898470 us; error 28 us; skew 500 ppm
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:06.900826 5796 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:06.901263 5796 webserver.cc:460] Webserver started at http://127.2.86.3:37689/ using document root <none> and password file <none>
I20260926 13:04:06.901506 5796 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:06.901573 5796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:06.901721 5796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:06.902750 5796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "5a03eb12995046d1af3e48cffe454c46"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "d91d6dfc4edd503c3d187393570bb8f5"
server_key_iv: "4fd5582bfd207ed79281876ea7e375ba"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.903100 5796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "5a03eb12995046d1af3e48cffe454c46"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "d91d6dfc4edd503c3d187393570bb8f5"
server_key_iv: "4fd5582bfd207ed79281876ea7e375ba"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.904374 5796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 13:04:06.905054 5811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.905220 5796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:06.905309 5796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "5a03eb12995046d1af3e48cffe454c46"
format_stamp: "Formatted at 2026-09-26 13:04:06 on dist-test-slave-j4lq"
server_key: "d91d6dfc4edd503c3d187393570bb8f5"
server_key_iv: "4fd5582bfd207ed79281876ea7e375ba"
server_key_version: "encryptionkey@0"
I20260926 13:04:06.905396 5796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:06.923949 5796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:06.926396 5796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:06.926555 5796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:06.926771 5796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:06.927089 5796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:06.927147 5796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.927223 5796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:06.927253 5796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:06.932957 5796 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:39785
I20260926 13:04:06.933015 5924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:39785 every 8 connection(s)
I20260926 13:04:06.933337 5796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:06.935674 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5796
I20260926 13:04:06.935781 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:06.935976 2392 external_mini_cluster.cc:1428] Setting key f33747d664f77a16173259b97d2192df
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:06.941941 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.934063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:47691 (local address 127.2.86.62:36441)
0926 13:04:06.934122 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:06.934124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:06.934324 (+ 200us) server_negotiation.cc:407] Connection header received
0926 13:04:06.934764 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:06.934765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:06.934775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:06.934826 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:06.935796 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.936402 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.937553 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.937735 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.937788 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.939703 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:06.939704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:06.939704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:06.939710 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:06.940796 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.941180 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.941181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.941181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.941229 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.941440 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.941440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.941441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.941537 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:06.941585 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:06.941808 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 13:04:06.941868 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:04:06.942103 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.934055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:36441 (local address 127.2.86.3:47691)
0926 13:04:06.934203 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:06.934237 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:06.934688 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:06.934883 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:06.934886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:06.935418 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:06.935698 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.935705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.936472 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.936473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.937483 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:06.937485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.937803 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.937803 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:06.937905 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.938305 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:06.938313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:06.939610 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:06.940900 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.940906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.940908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.941119 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.941288 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:06.941288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:06.941288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:06.941388 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:06.941593 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:06.941597 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:06.941703 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 13:04:06.941785 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:04:06.941899 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260926 13:04:06.942553 5925 heartbeater.cc:347] Connected to a master server at 127.2.86.62:36441
I20260926 13:04:06.942651 5925 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:06.942809 5925 heartbeater.cc:511] Master 127.2.86.62:36441 requested a full tablet report, sending...
I20260926 13:04:06.943173 5466 ts_manager.cc:194] Registered new tserver with Master: 5a03eb12995046d1af3e48cffe454c46 (127.2.86.3:39785)
I20260926 13:04:06.943504 5466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:47691
I20260926 13:04:06.947682 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 13:04:06 dist-test-slave-j4lq krb5kdc[5421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427846, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:06.969404 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:06.962847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50732 (local address 127.2.86.62:36441)
0926 13:04:06.962904 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:06.962906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:06.963006 (+ 100us) server_negotiation.cc:407] Connection header received
0926 13:04:06.963050 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:06.963051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:06.963059 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:06.963110 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:06.963647 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.964109 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.964783 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:06.965004 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:06.965054 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:06.967254 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:06.967255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:06.967256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:06.967262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:06.968424 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.968702 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.968702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.968703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.968747 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:06.969002 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:06.969002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:06.969002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:06.969129 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:06.969202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:06.969282 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 13:04:06.969343 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 13:04:06.969615 5466 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50732
I20260926 13:04:06.970135 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5528
I20260926 13:04:06.974903 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5662
I20260926 13:04:06.979337 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5796
I20260926 13:04:06.983189 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5434
2026-09-26T13:04:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1786 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:07 dist-test-slave-j4lq krb5kdc[5937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:07 dist-test-slave-j4lq krb5kdc[5937](info): set up 2 sockets
Sep 26 13:04:07 dist-test-slave-j4lq krb5kdc[5937](info): commencing operation
krb5kdc: starting...
W20260926 13:04:08.012830 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:08.124429 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39253
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:39253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:08.216166 5950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:08.216362 5950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:08.216384 5950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:08.217562 5950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:08.217600 5950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:08.217615 5950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:08.217628 5950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:08.217640 5950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:08.219036 5950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:39253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:08.219210 5950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:08.219414 5950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:08.224324 5958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.224341 5956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.224507 5950 server_base.cc:1034] running on GCE node
W20260926 13:04:08.224340 5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.224771 5950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:08.225085 5950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:08.226239 5950 hybrid_clock.cc:648] HybridClock initialized: now 1790427848226221 us; error 31 us; skew 500 ppm
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:08.228667 5950 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:08.229115 5950 webserver.cc:460] Webserver started at http://127.2.86.62:41799/ using document root <none> and password file <none>
I20260926 13:04:08.229362 5950 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:08.229404 5950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:08.229487 5950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:08.230403 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "6d40f526dab14cc390dc652b55ddac71"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "9396868a0ea2701565bd821e8da69cdc"
server_key_iv: "1f17459a2acbb00061f48e2e7e20b645"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.230679 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "6d40f526dab14cc390dc652b55ddac71"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "9396868a0ea2701565bd821e8da69cdc"
server_key_iv: "1f17459a2acbb00061f48e2e7e20b645"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.231890 5950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:08.232482 5965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.232666 5950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:08.232766 5950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "6d40f526dab14cc390dc652b55ddac71"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "9396868a0ea2701565bd821e8da69cdc"
server_key_iv: "1f17459a2acbb00061f48e2e7e20b645"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.232836 5950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:08.243655 5950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:08.245879 5950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:08.246037 5950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:08.249732 5950 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:39253
I20260926 13:04:08.249799 6017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:39253 every 8 connection(s)
I20260926 13:04:08.250443 5950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:08.251117 6018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:08.253263 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71: Bootstrap starting.
I20260926 13:04:08.253840 6018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:08.254117 6018 log.cc:824] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:08.254755 6018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71: No bootstrap required, opened a new log
I20260926 13:04:08.256258 6018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } }
I20260926 13:04:08.256412 6018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:08.256458 6018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d40f526dab14cc390dc652b55ddac71, State: Initialized, Role: FOLLOWER
I20260926 13:04:08.256621 6018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } }
I20260926 13:04:08.256717 6018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:08.256771 6018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:08.256819 6018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:08.257666 6018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } }
I20260926 13:04:08.257782 6018 leader_election.cc:302] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d40f526dab14cc390dc652b55ddac71; no voters:
I20260926 13:04:08.257937 6018 leader_election.cc:288] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:08.257963 6021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:08.258116 6021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [term 1 LEADER]: Becoming Leader. State: Replica: 6d40f526dab14cc390dc652b55ddac71, State: Running, Role: LEADER
I20260926 13:04:08.258177 6018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:08.258245 6021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } }
I20260926 13:04:08.258630 6023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d40f526dab14cc390dc652b55ddac71. Latest consensus state: current_term: 1 leader_uuid: "6d40f526dab14cc390dc652b55ddac71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } } }
I20260926 13:04:08.258751 6023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:08.258625 6022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d40f526dab14cc390dc652b55ddac71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d40f526dab14cc390dc652b55ddac71" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39253 } } }
I20260926 13:04:08.258805 6022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:08.258944 6030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:08.259483 6030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:08.259666 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 5950
I20260926 13:04:08.259768 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:08.259938 2392 external_mini_cluster.cc:1428] Setting key b9bcaca024885a3f4f97a834a78cb6f6
I20260926 13:04:08.260982 6030 catalog_manager.cc:1347] Generated new cluster ID: bed1caed90be481f9beb0c857ae5a44e
I20260926 13:04:08.261034 6030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:08.273833 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.260400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.2.86.62:39253)
0926 13:04:08.264961 (+ 4561us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.264973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.264998 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:04:08.265084 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.265089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.265130 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.265261 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.265872 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.267076 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.267825 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.268253 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.268377 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.270724 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.270728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.270729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.270742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.272326 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.272692 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.272695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.272696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.272777 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.273021 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.273022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.273022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.273238 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.273390 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.273494 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.273625 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4514,"thread_start_us":64,"threads_started":1}
I20260926 13:04:08.277437 6030 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:08.278100 6030 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:08.279981 6030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d40f526dab14cc390dc652b55ddac71: Generated new TSK 0
I20260926 13:04:08.280123 6030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:08.307493 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:08.400183 6044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:08.400365 6044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:08.400388 6044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:08.401619 6044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:08.401659 6044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:08.401685 6044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:08.403139 6044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:08.403365 6044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:08.403582 6044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:08.409178 6049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.409188 6050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.409341 6044 server_base.cc:1034] running on GCE node
W20260926 13:04:08.409439 6052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.409605 6044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:08.409849 6044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:08.411042 6044 hybrid_clock.cc:648] HybridClock initialized: now 1790427848410999 us; error 38 us; skew 500 ppm
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:08.422115 6044 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:08.422552 6044 webserver.cc:460] Webserver started at http://127.2.86.1:32883/ using document root <none> and password file <none>
I20260926 13:04:08.422734 6044 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:08.422782 6044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:08.422871 6044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:08.423825 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "e1fb1aac94db4b5f96e5a3eff4531b9f"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "aa2a2f7fbd17d43853cba0167e244814"
server_key_iv: "0666622192bc9aac36690c1e88305212"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.424111 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "e1fb1aac94db4b5f96e5a3eff4531b9f"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "aa2a2f7fbd17d43853cba0167e244814"
server_key_iv: "0666622192bc9aac36690c1e88305212"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.425379 6044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:08.426069 6059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.426270 6044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:08.426352 6044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "e1fb1aac94db4b5f96e5a3eff4531b9f"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "aa2a2f7fbd17d43853cba0167e244814"
server_key_iv: "0666622192bc9aac36690c1e88305212"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.426403 6044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:08.447621 6044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:08.449872 6044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:08.449980 6044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:08.450148 6044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:08.450450 6044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:08.450484 6044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.450527 6044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:08.450551 6044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.456341 6044 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:45917
I20260926 13:04:08.456390 6172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:45917 every 8 connection(s)
I20260926 13:04:08.456681 6044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:08.462903 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6044
I20260926 13:04:08.463011 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:08.463189 2392 external_mini_cluster.cc:1428] Setting key 80000555973dfe1279e18a3c540e623e
I20260926 13:04:08.465977 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.457468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:42847 (local address 127.2.86.62:39253)
0926 13:04:08.457513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.457515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.457738 (+ 223us) server_negotiation.cc:407] Connection header received
0926 13:04:08.458756 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.458758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.458771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.458811 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.459419 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.459967 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.460947 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.461141 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.461192 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.463133 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.463134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.463135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.463150 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.464565 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.465051 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.465052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.465053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.465111 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.465352 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.465353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.465353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.465462 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.465542 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.465798 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.465890 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:08.466168 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.457459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39253 (local address 127.2.86.1:42847)
0926 13:04:08.457621 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:08.457655 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:08.458638 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:08.458877 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:08.458881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:08.459113 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:08.459337 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.459341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.460043 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.460043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.460828 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.460830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.461211 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.461212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.461309 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.461728 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:08.461736 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:08.463031 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:08.464669 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.464676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.464678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.464966 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.465177 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.465178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.465178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.465286 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.465534 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:08.465541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:08.465673 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 13:04:08.465777 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 13:04:08.465919 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260926 13:04:08.466636 6173 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39253
I20260926 13:04:08.466737 6173 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:08.466902 6173 heartbeater.cc:511] Master 127.2.86.62:39253 requested a full tablet report, sending...
I20260926 13:04:08.467401 5982 ts_manager.cc:194] Registered new tserver with Master: e1fb1aac94db4b5f96e5a3eff4531b9f (127.2.86.1:45917)
I20260926 13:04:08.467998 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:42847
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:08.487040 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:08.565593 6178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:08.565774 6178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:08.565796 6178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:08.566963 6178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:08.567004 6178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:08.567030 6178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:08.568524 6178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:08.568754 6178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:08.569026 6178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:08.573953 6186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.573967 6183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.574034 6184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.574095 6178 server_base.cc:1034] running on GCE node
I20260926 13:04:08.574381 6178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:08.574635 6178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:08.575789 6178 hybrid_clock.cc:648] HybridClock initialized: now 1790427848575773 us; error 34 us; skew 500 ppm
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:08.578426 6178 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:08.578783 6178 webserver.cc:460] Webserver started at http://127.2.86.2:38039/ using document root <none> and password file <none>
I20260926 13:04:08.578967 6178 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:08.579005 6178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:08.579106 6178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:08.580050 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "57be305a420d4071941b6f00a1495b09"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "ecee4f4688457dff088e8c8b65a85ca1"
server_key_iv: "ae9bc2221197c18b851ff2d6c70468ac"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.580348 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "57be305a420d4071941b6f00a1495b09"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "ecee4f4688457dff088e8c8b65a85ca1"
server_key_iv: "ae9bc2221197c18b851ff2d6c70468ac"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.581710 6178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:08.582388 6193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.582547 6178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:08.582618 6178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "57be305a420d4071941b6f00a1495b09"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "ecee4f4688457dff088e8c8b65a85ca1"
server_key_iv: "ae9bc2221197c18b851ff2d6c70468ac"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.582669 6178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:08.597790 6178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:08.600107 6178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:08.600220 6178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:08.600411 6178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:08.600646 6178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:08.600674 6178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.600696 6178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:08.600709 6178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.606230 6178 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:39465
I20260926 13:04:08.606312 6306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:39465 every 8 connection(s)
I20260926 13:04:08.606611 6178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:08.611099 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6178
I20260926 13:04:08.611210 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:08.611428 2392 external_mini_cluster.cc:1428] Setting key c6c4656ca26f57d522a4a6a14f82768b
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:08.615948 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.607324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:50131 (local address 127.2.86.62:39253)
0926 13:04:08.607394 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.607396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.607580 (+ 184us) server_negotiation.cc:407] Connection header received
0926 13:04:08.608535 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.608536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.608549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.608606 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.609227 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.609734 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.610716 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.610921 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.611005 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.613272 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.613273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.613274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.613282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.614551 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.615020 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.615021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.615021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.615065 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.615342 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.615344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.615344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.615463 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.615514 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.615832 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.615890 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:04:08.616312 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.607306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39253 (local address 127.2.86.2:50131)
0926 13:04:08.607454 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:08.607496 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:08.608422 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:08.608675 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:08.608679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:08.608948 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:08.609129 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.609133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.609828 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.609830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.610624 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.610626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.611013 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.611014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.611133 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.611689 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:08.611698 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:08.613140 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:08.614630 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.614636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.614638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.614946 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.615130 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.615131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.615131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.615273 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.615529 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:08.615536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:08.615698 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 13:04:08.615816 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 13:04:08.615979 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260926 13:04:08.616948 6307 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39253
I20260926 13:04:08.617053 6307 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:08.617249 6307 heartbeater.cc:511] Master 127.2.86.62:39253 requested a full tablet report, sending...
I20260926 13:04:08.617642 5982 ts_manager.cc:194] Registered new tserver with Master: 57be305a420d4071941b6f00a1495b09 (127.2.86.2:39465)
I20260926 13:04:08.618023 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:50131
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:08.635586 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:08.712618 6312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:08.712832 6312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:08.712893 6312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:08.714052 6312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:08.714118 6312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:08.714161 6312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:08.715626 6312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:08.715878 6312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:08.716096 6312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:08.720846 6317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.720909 6318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:08.720855 6320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:08.720902 6312 server_base.cc:1034] running on GCE node
I20260926 13:04:08.721287 6312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:08.721511 6312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:08.722651 6312 hybrid_clock.cc:648] HybridClock initialized: now 1790427848722636 us; error 30 us; skew 500 ppm
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:08.725135 6312 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:08.725492 6312 webserver.cc:460] Webserver started at http://127.2.86.3:38309/ using document root <none> and password file <none>
I20260926 13:04:08.725659 6312 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:08.725697 6312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:08.725781 6312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:08.726688 6312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "641bf5591f304ce295f22a83277bcbd8"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "e8852dbf24a44439527499d459fc2e94"
server_key_iv: "9b8dc051d197afe55d0987270587208e"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.726981 6312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "641bf5591f304ce295f22a83277bcbd8"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "e8852dbf24a44439527499d459fc2e94"
server_key_iv: "9b8dc051d197afe55d0987270587208e"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.728161 6312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:08.728811 6327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.729002 6312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:08.729096 6312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "641bf5591f304ce295f22a83277bcbd8"
format_stamp: "Formatted at 2026-09-26 13:04:08 on dist-test-slave-j4lq"
server_key: "e8852dbf24a44439527499d459fc2e94"
server_key_iv: "9b8dc051d197afe55d0987270587208e"
server_key_version: "encryptionkey@0"
I20260926 13:04:08.729176 6312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:08.746455 6312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:08.748334 6312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:08.748440 6312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:08.748602 6312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:08.748823 6312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:08.748850 6312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.748916 6312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:08.748950 6312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:08.754412 6312 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:42871
I20260926 13:04:08.754465 6440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:42871 every 8 connection(s)
I20260926 13:04:08.754761 6312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:08.760197 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6312
I20260926 13:04:08.760298 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:08.760516 2392 external_mini_cluster.cc:1428] Setting key c2af07950e8e6e13785eb3fe73d604be
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:08.764178 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.755449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:47593 (local address 127.2.86.62:39253)
0926 13:04:08.755523 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.755525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.755728 (+ 203us) server_negotiation.cc:407] Connection header received
0926 13:04:08.756617 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.756618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.756627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.756679 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.757329 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.757880 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.758997 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.759220 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.759274 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.761749 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.761750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.761750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.761757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.762823 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.763209 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.763211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.763211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.763270 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.763532 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.763533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.763533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.763651 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.763755 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.764015 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.764087 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:04:08.764361 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.755440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39253 (local address 127.2.86.3:47593)
0926 13:04:08.755605 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:08.755645 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:08.756529 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:08.756750 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:08.756753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:08.757073 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:08.757256 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.757260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.757952 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.757953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.758878 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:08.758880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.759289 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.759289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:08.759384 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.759935 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:08.759943 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:08.761651 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:08.762912 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.762918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.762920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.763124 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.763334 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:08.763335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:08.763335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:08.763465 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:08.763746 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:08.763752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:08.763884 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 13:04:08.763995 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 13:04:08.764141 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260926 13:04:08.764775 6441 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39253
I20260926 13:04:08.764915 6441 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:08.765074 6441 heartbeater.cc:511] Master 127.2.86.62:39253 requested a full tablet report, sending...
I20260926 13:04:08.765398 5982 ts_manager.cc:194] Registered new tserver with Master: 641bf5591f304ce295f22a83277bcbd8 (127.2.86.3:42871)
I20260926 13:04:08.765738 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:47593
I20260926 13:04:08.772104 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:04:08.793669 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.786825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.2.86.1:45917)
0926 13:04:08.786974 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.786978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.786989 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:04:08.787012 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.787013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.787026 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.787092 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.787612 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.788270 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.789059 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.789361 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.789417 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.791388 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.791394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.791394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.791401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.792707 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.793049 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.793052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.793052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.793105 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.793266 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.793267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.793267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.793385 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.793479 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.793539 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.793597 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:08.801218 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:08.794799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.2.86.62:39253)
0926 13:04:08.794845 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:08.794847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:08.794911 (+ 64us) server_negotiation.cc:407] Connection header received
0926 13:04:08.794983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:08.794984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:08.794993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:08.795062 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:08.795497 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.795960 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.796628 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:08.796825 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:08.796926 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:08.798898 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:08.798899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:08.798899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:08.798906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:08.800023 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.800369 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.800370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.800370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.800424 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:08.800616 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:08.800617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:08.800617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:08.800751 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:08.800850 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:08.801013 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 13:04:08.801122 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 13:04:08.801406 5982 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56298
I20260926 13:04:08.801692 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6044
I20260926 13:04:08.806135 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6178
I20260926 13:04:08.810262 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6312
I20260926 13:04:08.814412 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 5950
2026-09-26T13:04:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1831 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[6453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[6453](info): set up 2 sockets
Sep 26 13:04:08 dist-test-slave-j4lq krb5kdc[6453](info): commencing operation
krb5kdc: starting...
W20260926 13:04:09.828858 2392 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:09 dist-test-slave-j4lq krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:09.945807 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43833
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:10.023432 6466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.023633 6466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.023653 6466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.024691 6466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:10.024724 6466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.024737 6466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.024749 6466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:10.024761 6466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:10.026161 6466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:43833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.026345 6466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.026505 6466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.030997 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.031056 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.031008 6474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.031225 6466 server_base.cc:1034] running on GCE node
I20260926 13:04:10.031440 6466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.031658 6466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.032833 6466 hybrid_clock.cc:648] HybridClock initialized: now 1790427850032824 us; error 26 us; skew 500 ppm
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:10.035174 6466 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:10.035517 6466 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.035564 6466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.035650 6466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.036595 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "f4c46fbf3dd747fbae8ce0021e3a419f"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "3c88897861fc2120863af4beb41e688c"
server_key_iv: "b7c28409d41b721daea60a0dc72052f1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.036967 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "f4c46fbf3dd747fbae8ce0021e3a419f"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "3c88897861fc2120863af4beb41e688c"
server_key_iv: "b7c28409d41b721daea60a0dc72052f1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.038157 6466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:10.038787 6480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.038932 6466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:10.039022 6466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "f4c46fbf3dd747fbae8ce0021e3a419f"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "3c88897861fc2120863af4beb41e688c"
server_key_iv: "b7c28409d41b721daea60a0dc72052f1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.039072 6466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.053418 6466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.055109 6466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.055214 6466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.058866 6466 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:43833
I20260926 13:04:10.058936 6532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:43833 every 8 connection(s)
I20260926 13:04:10.059193 6466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:10.060047 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6466
I20260926 13:04:10.060155 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:10.060217 6533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:10.060359 2392 external_mini_cluster.cc:1428] Setting key 16a2a3524bd60b0aac10de949e3442a6
I20260926 13:04:10.062995 6533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: Bootstrap starting.
I20260926 13:04:10.063621 6533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:10.063908 6533 log.cc:824] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:10.064563 6533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: No bootstrap required, opened a new log
I20260926 13:04:10.066499 6533 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } }
I20260926 13:04:10.066658 6533 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:10.066723 6533 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c46fbf3dd747fbae8ce0021e3a419f, State: Initialized, Role: FOLLOWER
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:10.066888 6533 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } }
I20260926 13:04:10.066977 6533 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:10.067024 6533 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:10.067065 6533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:10.067732 6533 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } }
I20260926 13:04:10.067838 6533 leader_election.cc:302] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4c46fbf3dd747fbae8ce0021e3a419f; no voters:
I20260926 13:04:10.067996 6533 leader_election.cc:288] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:10.068025 6538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:10.068133 6538 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [term 1 LEADER]: Becoming Leader. State: Replica: f4c46fbf3dd747fbae8ce0021e3a419f, State: Running, Role: LEADER
I20260926 13:04:10.068210 6538 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } }
I20260926 13:04:10.068259 6533 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:10.069628 6539 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } } }
I20260926 13:04:10.069744 6539 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:10.069890 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4c46fbf3dd747fbae8ce0021e3a419f. Latest consensus state: current_term: 1 leader_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c46fbf3dd747fbae8ce0021e3a419f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43833 } } }
I20260926 13:04:10.069945 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:10.070019 6551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:10.070552 6551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:10.071282 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.060848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.2.86.62:43833)
0926 13:04:10.061077 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.061085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.061108 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:04:10.061641 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.061648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.061699 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.061847 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.062629 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.063593 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.064413 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.064859 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.065002 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.067454 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.067458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.067458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:10.067468 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.069519 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.069979 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.069983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.069983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.070093 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.070377 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.070378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.070378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.070685 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.070824 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.070954 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.071087 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260926 13:04:10.072108 6551 catalog_manager.cc:1347] Generated new cluster ID: 49a29523b6d94b2794dbc4e2536b04bd
I20260926 13:04:10.072154 6551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:10.085530 6551 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:10.086272 6551 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:10.092906 6551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: Generated new TSK 0
I20260926 13:04:10.093066 6551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:10.123719 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:10.202431 6559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.202653 6559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.202692 6559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.203847 6559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.203912 6559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.203953 6559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:10.205961 6559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.206219 6559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.206460 6559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.211308 6564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.211311 6567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.211330 6565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.211687 6559 server_base.cc:1034] running on GCE node
I20260926 13:04:10.211911 6559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.212143 6559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.213294 6559 hybrid_clock.cc:648] HybridClock initialized: now 1790427850213294 us; error 36 us; skew 500 ppm
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:10.215715 6559 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:10.216081 6559 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.216128 6559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.216218 6559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.217257 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "0206ab3b860647b7aa01d5bc6d05db59"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "69466c446db4be934c256b7c9fc88f5f"
server_key_iv: "aede230ef4c8a3bee56f8cd2839ca93e"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.217557 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "0206ab3b860647b7aa01d5bc6d05db59"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "69466c446db4be934c256b7c9fc88f5f"
server_key_iv: "aede230ef4c8a3bee56f8cd2839ca93e"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.218709 6559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:10.219336 6573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.219542 6559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:10.219625 6559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "0206ab3b860647b7aa01d5bc6d05db59"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "69466c446db4be934c256b7c9fc88f5f"
server_key_iv: "aede230ef4c8a3bee56f8cd2839ca93e"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.219676 6559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.245062 6559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.246795 6559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.246899 6559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.247057 6559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:10.247349 6559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:10.247380 6559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.247434 6559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:10.247466 6559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.253111 6559 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:34899
I20260926 13:04:10.253151 6686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:34899 every 8 connection(s)
I20260926 13:04:10.253394 6559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:10.258394 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6559
I20260926 13:04:10.258483 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:10.258723 2392 external_mini_cluster.cc:1428] Setting key 436c466e479e94b9660f4156b5e2a575
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:10.262080 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.254021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:37503 (local address 127.2.86.62:43833)
0926 13:04:10.254113 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.254115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.254310 (+ 195us) server_negotiation.cc:407] Connection header received
0926 13:04:10.254740 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.254741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.254751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.254799 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.255610 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.256090 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.257048 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.257237 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.257287 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.259451 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.259453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.259453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:10.259460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.260732 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.261153 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.261154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.261155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.261209 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.261460 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.261461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.261461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.261568 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.261639 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.261905 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.261992 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 13:04:10.262259 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.253996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43833 (local address 127.2.86.1:37503)
0926 13:04:10.254189 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.254222 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.254663 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.254849 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.254853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.255078 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:10.255538 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.255542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.256161 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.256166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.256980 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.256982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.257303 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.257304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.257390 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.257796 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:10.257803 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:10.259339 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:10.260814 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.260821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.260823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.261070 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.261275 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.261276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.261277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.261387 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.261640 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:10.261647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:10.261777 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.261874 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.262029 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260926 13:04:10.262686 6687 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43833
I20260926 13:04:10.262771 6687 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:10.262924 6687 heartbeater.cc:511] Master 127.2.86.62:43833 requested a full tablet report, sending...
I20260926 13:04:10.263466 6497 ts_manager.cc:194] Registered new tserver with Master: 0206ab3b860647b7aa01d5bc6d05db59 (127.2.86.1:34899)
I20260926 13:04:10.264439 6497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:37503
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:10.281641 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:10.358994 6692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.359201 6692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.359241 6692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.360374 6692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.360435 6692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.360496 6692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:10.362056 6692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.362308 6692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.362527 6692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.367244 6698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.367259 6700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.367395 6692 server_base.cc:1034] running on GCE node
W20260926 13:04:10.367262 6697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.367695 6692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.367879 6692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.369042 6692 hybrid_clock.cc:648] HybridClock initialized: now 1790427850369028 us; error 31 us; skew 500 ppm
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:10.371518 6692 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:10.371835 6692 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.371877 6692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.371955 6692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.372851 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "ecae08d4e7aa43508139709044721b23"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "fc1337878aace3b3c0969f1a315a7543"
server_key_iv: "c069abc426b853f4498e885c11be0dc1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.373170 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "ecae08d4e7aa43508139709044721b23"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "fc1337878aace3b3c0969f1a315a7543"
server_key_iv: "c069abc426b853f4498e885c11be0dc1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.374379 6692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:10.375000 6706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.375167 6692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:10.375229 6692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "ecae08d4e7aa43508139709044721b23"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "fc1337878aace3b3c0969f1a315a7543"
server_key_iv: "c069abc426b853f4498e885c11be0dc1"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.375311 6692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.397253 6692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.398998 6692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.399148 6692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.399374 6692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:10.399742 6692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:10.399806 6692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.399839 6692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:10.399866 6692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.405467 6692 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:45081
I20260926 13:04:10.405490 6819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:45081 every 8 connection(s)
I20260926 13:04:10.405754 6692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:10.405925 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6692
I20260926 13:04:10.406028 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:10.406275 2392 external_mini_cluster.cc:1428] Setting key d6391dada086c999eabcb5301b705f69
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
I20260926 13:04:10.416117 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.406885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:56593 (local address 127.2.86.62:43833)
0926 13:04:10.406924 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.406926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.406961 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:04:10.407457 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.407459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.407471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.407541 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.408514 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.409139 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.410708 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.410893 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.410957 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.413319 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.413320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.413321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:10.413328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.414749 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.415131 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.415132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.415132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.415189 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.415453 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.415453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.415454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.415559 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.415650 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.415936 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.416026 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:04:10.416303 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.406681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43833 (local address 127.2.86.2:56593)
0926 13:04:10.406844 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.406883 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.407344 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.407591 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.407594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.407934 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:10.408415 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.408420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.409227 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.409229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.410617 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.410620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.410980 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.410981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.411129 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.411622 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:10.411633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:10.413228 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:10.414837 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.414843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.414845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.415057 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.415253 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.415254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.415255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.415371 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.415640 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:10.415646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:10.415791 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.415916 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.416064 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260926 13:04:10.416857 6820 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43833
I20260926 13:04:10.417001 6820 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
I20260926 13:04:10.417150 6820 heartbeater.cc:511] Master 127.2.86.62:43833 requested a full tablet report, sending...
I20260926 13:04:10.417546 6497 ts_manager.cc:194] Registered new tserver with Master: ecae08d4e7aa43508139709044721b23 (127.2.86.2:45081)
I20260926 13:04:10.417923 6497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:56593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:10.429903 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:10.509092 6825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.509287 6825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.509310 6825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.510449 6825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.510491 6825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.510545 6825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:10.511960 6825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.512183 6825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.512391 6825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.517323 6830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.517328 6831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.517485 6825 server_base.cc:1034] running on GCE node
W20260926 13:04:10.517336 6833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.517796 6825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.518062 6825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.519217 6825 hybrid_clock.cc:648] HybridClock initialized: now 1790427850519197 us; error 33 us; skew 500 ppm
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:10.521685 6825 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:10.522007 6825 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.522074 6825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.522297 6825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.523415 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "a8875be804714a8184508f5aa35602c9"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "601c64527508ac95ac8333c71377612b"
server_key_iv: "fb2f8a866ecfe2a5da3b11ee9fcf01d8"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.523813 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "a8875be804714a8184508f5aa35602c9"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "601c64527508ac95ac8333c71377612b"
server_key_iv: "fb2f8a866ecfe2a5da3b11ee9fcf01d8"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.525111 6825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:10.525775 6839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.525944 6825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:10.526032 6825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a8875be804714a8184508f5aa35602c9"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "601c64527508ac95ac8333c71377612b"
server_key_iv: "fb2f8a866ecfe2a5da3b11ee9fcf01d8"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.526109 6825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.533812 6825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.536003 6825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.536155 6825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.536391 6825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:10.536680 6825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:10.536733 6825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.536798 6825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:10.536829 6825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.542421 6825 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:36921
I20260926 13:04:10.542496 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:36921 every 8 connection(s)
I20260926 13:04:10.542727 6825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:10.544028 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6825
I20260926 13:04:10.544140 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:10.544354 2392 external_mini_cluster.cc:1428] Setting key 4a364e785f2286bf86a919ed395d4b01
Sep 26 13:04:10 dist-test-slave-j4lq krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427850, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:10.551200 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.543360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:47339 (local address 127.2.86.62:43833)
0926 13:04:10.543431 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.543433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.543619 (+ 186us) server_negotiation.cc:407] Connection header received
0926 13:04:10.544125 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.544127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.544138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.544203 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.545394 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.545879 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.546814 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.546989 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.547041 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.548831 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.548833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.548833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:10.548839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.550137 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.550491 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.550492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.550493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.550538 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.550736 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.550736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.550737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.550826 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.550873 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.551089 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.551142 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:04:10.551342 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.543345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:43833 (local address 127.2.86.3:47339)
0926 13:04:10.543500 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.543533 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.544015 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.544271 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.544276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.544595 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:10.545280 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.545285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.545947 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.545947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.546718 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.546720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.547058 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.547059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.547150 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.547527 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:10.547534 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:10.548755 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:10.550206 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.550211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.550213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.550426 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.550594 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:10.550595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:10.550595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:10.550684 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:10.550877 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:10.550881 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:10.550985 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.551061 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.551166 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260926 13:04:10.551745 6953 heartbeater.cc:347] Connected to a master server at 127.2.86.62:43833
I20260926 13:04:10.551826 6953 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:10.551930 6953 heartbeater.cc:511] Master 127.2.86.62:43833 requested a full tablet report, sending...
I20260926 13:04:10.552274 6497 ts_manager.cc:194] Registered new tserver with Master: a8875be804714a8184508f5aa35602c9 (127.2.86.3:36921)
I20260926 13:04:10.552591 6497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:47339
I20260926 13:04:10.556085 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:04:10.562544 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.557296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.2.86.62:43833)
0926 13:04:10.557361 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.557362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.557404 (+ 42us) server_negotiation.cc:407] Connection header received
0926 13:04:10.557457 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.557458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.557465 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.557507 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.558009 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.558459 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.559109 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.559273 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.559318 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.560303 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.560304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.560304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:10.560310 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.561661 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.561932 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.561933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.561933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.562015 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:10.562153 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:10.562153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:10.562153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:10.562260 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.562325 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.562411 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.562458 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:04:10.564023 6497 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:10.565609 6497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:10.571770 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.567894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:34899 (local address 127.0.0.1:60388)
0926 13:04:10.568171 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.568212 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.568264 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.568485 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.568486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.568489 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:10.568529 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.568530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569720 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.569721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.570728 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.570730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571507 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.571585 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571598 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.571656 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.571701 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":35,"threads_started":1}
I20260926 13:04:10.571875 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.568026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.2.86.3:36921)
0926 13:04:10.568210 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.568214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.568223 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:10.568255 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.568256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.568272 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.568468 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:10.568524 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569654 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.570905 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571524 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571611 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571660 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.571759 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.571796 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260926 13:04:10.572057 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.568010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.2.86.1:34899)
0926 13:04:10.568149 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.568154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.568255 (+ 101us) server_negotiation.cc:407] Connection header received
0926 13:04:10.568326 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.568328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.568342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.568410 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:10.568642 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569628 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.570815 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571426 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571497 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571546 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.571960 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.571998 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260926 13:04:10.571871 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.567772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:36921 (local address 127.0.0.1:58474)
0926 13:04:10.567852 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.567869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.567923 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.568362 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.568362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.568365 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:10.568404 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.568405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569763 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.569764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.570833 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.570834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571621 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.571686 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571691 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.571747 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.571784 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":48,"thread_start_us":48,"threads_started":1}
I20260926 13:04:10.572172 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.567566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:45081 (local address 127.0.0.1:52466)
0926 13:04:10.567675 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.567709 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.567762 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.568058 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.568061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.568063 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:10.568121 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.568122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569438 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.569439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.570958 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.570960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571765 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.571812 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571814 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.572073 (+ 259us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.572102 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":86,"thread_start_us":71,"threads_started":1}
I20260926 13:04:10.572372 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.567665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.2.86.2:45081)
0926 13:04:10.567813 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.567817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.567828 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:04:10.567869 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.567871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.567889 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.567964 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:10.568209 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.569336 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571029 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.571639 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.571692 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.571736 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.572234 (+ 498us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.572270 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260926 13:04:10.572789 6887 tablet_service.cc:1467] Processing CreateTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e (DEFAULT_TABLE table=test-table [id=3186d0c716e948f28946478bc0d9b959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:10.573123 6887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5006b2bb8da4ed1a1b6db210c6eb16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:10.573226 6754 tablet_service.cc:1467] Processing CreateTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e (DEFAULT_TABLE table=test-table [id=3186d0c716e948f28946478bc0d9b959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:10.573500 6754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5006b2bb8da4ed1a1b6db210c6eb16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:10.575673 6967 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Bootstrap starting.
I20260926 13:04:10.576026 6968 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Bootstrap starting.
I20260926 13:04:10.576277 6967 tablet_bootstrap.cc:654] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:10.576414 6621 tablet_service.cc:1467] Processing CreateTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e (DEFAULT_TABLE table=test-table [id=3186d0c716e948f28946478bc0d9b959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:10.576715 6968 tablet_bootstrap.cc:654] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:10.576776 6621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5006b2bb8da4ed1a1b6db210c6eb16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:10.576926 6967 log.cc:824] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:10.577399 6968 log.cc:824] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:10.577730 6967 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: No bootstrap required, opened a new log
I20260926 13:04:10.577895 6967 ts_tablet_manager.cc:1397] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:10.578115 6968 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: No bootstrap required, opened a new log
I20260926 13:04:10.578197 6968 ts_tablet_manager.cc:1397] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 13:04:10.579710 6968 raft_consensus.cc:348] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.579895 6968 raft_consensus.cc:374] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:10.579937 6968 raft_consensus.cc:729] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecae08d4e7aa43508139709044721b23, State: Initialized, Role: FOLLOWER
I20260926 13:04:10.579882 6971 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: Bootstrap starting.
I20260926 13:04:10.579939 6967 raft_consensus.cc:348] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.580044 6968 consensus_queue.cc:260] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.580106 6967 raft_consensus.cc:374] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:10.580145 6967 raft_consensus.cc:729] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8875be804714a8184508f5aa35602c9, State: Initialized, Role: FOLLOWER
I20260926 13:04:10.580302 6968 ts_tablet_manager.cc:1428] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:04:10.580277 6967 consensus_queue.cc:260] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.580313 6820 heartbeater.cc:503] Master 127.2.86.62:43833 was elected leader, sending a full tablet report...
I20260926 13:04:10.580603 6971 tablet_bootstrap.cc:654] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:10.580575 6953 heartbeater.cc:503] Master 127.2.86.62:43833 was elected leader, sending a full tablet report...
I20260926 13:04:10.580571 6967 ts_tablet_manager.cc:1428] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:04:10.581187 6971 log.cc:824] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:10.582041 6971 tablet_bootstrap.cc:492] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: No bootstrap required, opened a new log
I20260926 13:04:10.582089 6971 ts_tablet_manager.cc:1397] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 13:04:10.583570 6971 raft_consensus.cc:348] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.583683 6971 raft_consensus.cc:374] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:10.583702 6971 raft_consensus.cc:729] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0206ab3b860647b7aa01d5bc6d05db59, State: Initialized, Role: FOLLOWER
I20260926 13:04:10.583833 6971 consensus_queue.cc:260] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.584082 6971 ts_tablet_manager.cc:1428] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:04:10.584118 6687 heartbeater.cc:503] Master 127.2.86.62:43833 was elected leader, sending a full tablet report...
I20260926 13:04:10.618362 6972 raft_consensus.cc:482] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:10.618466 6972 raft_consensus.cc:504] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.618738 6972 leader_election.cc:288] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8875be804714a8184508f5aa35602c9 (127.2.86.3:36921), 0206ab3b860647b7aa01d5bc6d05db59 (127.2.86.1:34899)
I20260926 13:04:10.622452 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.619133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:39407 (local address 127.2.86.1:34899)
0926 13:04:10.619180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.619182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.619254 (+ 72us) server_negotiation.cc:407] Connection header received
0926 13:04:10.619318 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.619319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.619330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.619393 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:10.619527 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.620184 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.621537 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.622100 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.622175 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.622185 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.622346 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.622391 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:04:10.622452 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.619004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:39345 (local address 127.2.86.3:36921)
0926 13:04:10.619058 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.619060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.619070 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:10.619097 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.619098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.619114 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.619180 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:10.619321 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.620091 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.621537 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.622103 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.622175 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.622185 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.622346 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.622383 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 13:04:10.622458 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.618917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:36921 (local address 127.2.86.2:39345)
0926 13:04:10.618973 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.618985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.619027 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.619194 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.619195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.619196 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:10.619254 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.619255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.620167 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.620167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.621455 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.621456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.622182 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.622183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.622275 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.622277 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.622329 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.622365 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":49}
I20260926 13:04:10.622464 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.619044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:34899 (local address 127.2.86.2:39407)
0926 13:04:10.619196 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.619205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.619255 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.619408 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.619409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.619410 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:10.619459 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.619460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.620254 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.620255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.621452 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:10.621455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.622183 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.622183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:10.622258 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.622264 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.622323 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.622365 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260926 13:04:10.622808 6641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5006b2bb8da4ed1a1b6db210c6eb16e" candidate_uuid: "ecae08d4e7aa43508139709044721b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0206ab3b860647b7aa01d5bc6d05db59" is_pre_election: true
I20260926 13:04:10.622804 6907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5006b2bb8da4ed1a1b6db210c6eb16e" candidate_uuid: "ecae08d4e7aa43508139709044721b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8875be804714a8184508f5aa35602c9" is_pre_election: true
I20260926 13:04:10.622972 6907 raft_consensus.cc:2454] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecae08d4e7aa43508139709044721b23 in term 0.
I20260926 13:04:10.622977 6641 raft_consensus.cc:2454] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecae08d4e7aa43508139709044721b23 in term 0.
I20260926 13:04:10.623195 6707 leader_election.cc:302] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8875be804714a8184508f5aa35602c9, ecae08d4e7aa43508139709044721b23; no voters:
I20260926 13:04:10.623322 6972 raft_consensus.cc:2781] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:10.623409 6972 raft_consensus.cc:482] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:10.623452 6972 raft_consensus.cc:3037] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:10.624111 6972 raft_consensus.cc:504] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.624244 6972 leader_election.cc:288] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [CANDIDATE]: Term 1 election: Requested vote from peers a8875be804714a8184508f5aa35602c9 (127.2.86.3:36921), 0206ab3b860647b7aa01d5bc6d05db59 (127.2.86.1:34899)
I20260926 13:04:10.624426 6641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5006b2bb8da4ed1a1b6db210c6eb16e" candidate_uuid: "ecae08d4e7aa43508139709044721b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0206ab3b860647b7aa01d5bc6d05db59"
I20260926 13:04:10.624532 6641 raft_consensus.cc:3037] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:10.625273 6641 raft_consensus.cc:2454] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecae08d4e7aa43508139709044721b23 in term 1.
I20260926 13:04:10.625389 6907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5006b2bb8da4ed1a1b6db210c6eb16e" candidate_uuid: "ecae08d4e7aa43508139709044721b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8875be804714a8184508f5aa35602c9"
I20260926 13:04:10.625460 6707 leader_election.cc:302] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0206ab3b860647b7aa01d5bc6d05db59, ecae08d4e7aa43508139709044721b23; no voters:
I20260926 13:04:10.625476 6907 raft_consensus.cc:3037] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:10.625568 6972 raft_consensus.cc:2781] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:10.625767 6972 raft_consensus.cc:686] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 1 LEADER]: Becoming Leader. State: Replica: ecae08d4e7aa43508139709044721b23, State: Running, Role: LEADER
I20260926 13:04:10.625865 6972 consensus_queue.cc:237] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } }
I20260926 13:04:10.626253 6907 raft_consensus.cc:2454] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecae08d4e7aa43508139709044721b23 in term 1.
I20260926 13:04:10.626658 6496 catalog_manager.cc:5495] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecae08d4e7aa43508139709044721b23 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "ecae08d4e7aa43508139709044721b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecae08d4e7aa43508139709044721b23" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 45081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:10.643481 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.641416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.2.86.2:45081)
0926 13:04:10.641471 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.641473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.641481 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:10.641510 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.641511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.641534 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.641588 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:10.641680 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.642168 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.642833 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:10.643004 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:10.643053 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:10.643171 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:10.643254 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:10.643305 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.643338 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.643352 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:04:10.644975 6641 raft_consensus.cc:1261] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 1 FOLLOWER]: Refusing update from remote peer ecae08d4e7aa43508139709044721b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:10.645010 6907 raft_consensus.cc:1261] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 1 FOLLOWER]: Refusing update from remote peer ecae08d4e7aa43508139709044721b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:10.645283 6972 consensus_queue.cc:1035] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8875be804714a8184508f5aa35602c9" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 36921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:10.645396 6977 consensus_queue.cc:1035] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0206ab3b860647b7aa01d5bc6d05db59" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 34899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:10.647980 6981 mvcc.cc:205] Tried to move back new op lower bound from 7333592476239273984 to 7333592476164128768. Current Snapshot: MvccSnapshot[applied={T|T < 7333592476239273984 or (T in {7333592476239273984})}]
I20260926 13:04:10.648093 6980 mvcc.cc:205] Tried to move back new op lower bound from 7333592476239273984 to 7333592476164128768. Current Snapshot: MvccSnapshot[applied={T|T < 7333592476239273984}]
I20260926 13:04:10.648166 6982 mvcc.cc:205] Tried to move back new op lower bound from 7333592476239273984 to 7333592476164128768. Current Snapshot: MvccSnapshot[applied={T|T < 7333592476239273984}]
I20260926 13:04:10.650087 6496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:10.650192 6496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:10.650895 6496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4c46fbf3dd747fbae8ce0021e3a419f: Sending DeleteTablet for 3 replicas of tablet a5006b2bb8da4ed1a1b6db210c6eb16e
I20260926 13:04:10.651168 6754 tablet_service.cc:1514] Processing DeleteTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:10 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:52466
I20260926 13:04:10.651227 6887 tablet_service.cc:1514] Processing DeleteTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:10 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:58474
I20260926 13:04:10.651265 6621 tablet_service.cc:1514] Processing DeleteTablet for tablet a5006b2bb8da4ed1a1b6db210c6eb16e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:10 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:60388
I20260926 13:04:10.651437 6990 tablet_replica.cc:330] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: stopping tablet replica
I20260926 13:04:10.651463 6991 tablet_replica.cc:330] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59: stopping tablet replica
I20260926 13:04:10.651522 6990 raft_consensus.cc:2229] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:10.651542 6991 raft_consensus.cc:2229] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:10.651611 6990 raft_consensus.cc:2258] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:10.651626 6991 raft_consensus.cc:2258] T a5006b2bb8da4ed1a1b6db210c6eb16e P 0206ab3b860647b7aa01d5bc6d05db59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:10.651803 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6559
I20260926 13:04:10.653163 6989 tablet_replica.cc:330] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: stopping tablet replica
I20260926 13:04:10.653280 6989 raft_consensus.cc:2229] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:10.653398 6989 raft_consensus.cc:2258] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:10.653780 6989 ts_tablet_manager.cc:1905] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:10.655212 6989 ts_tablet_manager.cc:1918] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:10.655294 6989 log.cc:1196] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/a5006b2bb8da4ed1a1b6db210c6eb16e
I20260926 13:04:10.655534 6989 ts_tablet_manager.cc:1939] T a5006b2bb8da4ed1a1b6db210c6eb16e P ecae08d4e7aa43508139709044721b23: Deleting consensus metadata
I20260926 13:04:10.655886 6481 catalog_manager.cc:4841] TS ecae08d4e7aa43508139709044721b23 (127.2.86.2:45081): tablet a5006b2bb8da4ed1a1b6db210c6eb16e (table test-table [id=3186d0c716e948f28946478bc0d9b959]) successfully deleted
W20260926 13:04:10.657351 6481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:34899 (error 108)
I20260926 13:04:10.657603 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6692
I20260926 13:04:10.658320 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.657682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:34899 (local address 127.0.0.1:60394)
0926 13:04:10.658259 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.658295 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:34899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558}
W20260926 13:04:10.658463 6481 catalog_manager.cc:4584] TS 0206ab3b860647b7aa01d5bc6d05db59 (127.2.86.1:34899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5006b2bb8da4ed1a1b6db210c6eb16e: Network error: Client connection negotiation failed: client connection to 127.2.86.1:34899: connect: Connection refused (error 111)
I20260926 13:04:10.661322 6990 ts_tablet_manager.cc:1905] T a5006b2bb8da4ed1a1b6db210c6eb16e P a8875be804714a8184508f5aa35602c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:10.662235 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6825
I20260926 13:04:10.666815 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6466
I20260926 13:04:10.666854 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.666725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:36921 (local address 127.0.0.1:58482)
0926 13:04:10.666790 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.666819 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.3:36921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
2026-09-26T13:04:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1852 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T13:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:10Z Disabled control of system clock
I20260926 13:04:10.693074 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33993
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:04:10.770660 6996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.770890 6996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.770932 6996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.772019 6996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:10.772054 6996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.772066 6996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.772078 6996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:10.772090 6996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:10.773437 6996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.773623 6996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.773845 6996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.778599 7002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.778594 7004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.778577 7001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.778808 6996 server_base.cc:1034] running on GCE node
I20260926 13:04:10.779052 6996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.779371 6996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.780537 6996 hybrid_clock.cc:648] HybridClock initialized: now 1790427850780523 us; error 29 us; skew 500 ppm
I20260926 13:04:10.781890 6996 webserver.cc:460] Webserver started at http://127.2.86.62:34291/ using document root <none> and password file <none>
I20260926 13:04:10.782125 6996 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.782193 6996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.782368 6996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.783396 6996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "51eaef3e1921403d8174c6803d1a3ceb"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "f829a8473cd04990952fa7f54660e57c"
server_key_iv: "0b2b50c64e2379c4e7829731ef7f875d"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.783793 6996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "51eaef3e1921403d8174c6803d1a3ceb"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "f829a8473cd04990952fa7f54660e57c"
server_key_iv: "0b2b50c64e2379c4e7829731ef7f875d"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.785157 6996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:10.785880 7010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.786053 6996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:10.786156 6996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "51eaef3e1921403d8174c6803d1a3ceb"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "f829a8473cd04990952fa7f54660e57c"
server_key_iv: "0b2b50c64e2379c4e7829731ef7f875d"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.786235 6996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.789433 6996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.789681 6996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.789824 6996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.793437 6996 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33993
I20260926 13:04:10.793465 7062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33993 every 8 connection(s)
I20260926 13:04:10.793764 6996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:10.794366 7063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:10.796677 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 6996
I20260926 13:04:10.796799 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:10.796990 2392 external_mini_cluster.cc:1428] Setting key d203826d16fa63babf058ddf6c4acf56
I20260926 13:04:10.797047 7063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: Bootstrap starting.
I20260926 13:04:10.797791 7063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:10.798136 7063 log.cc:824] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:10.798885 7063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: No bootstrap required, opened a new log
I20260926 13:04:10.799541 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.797464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.2.86.62:33993)
0926 13:04:10.797660 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.797669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.797701 (+ 32us) server_negotiation.cc:407] Connection header received
0926 13:04:10.798308 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.798318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.798367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.798493 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.799108 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.799111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.799111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:10.799116 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.799163 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:10.799175 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.799263 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.799302 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.799355 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260926 13:04:10.800592 7063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } }
I20260926 13:04:10.800716 7063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:10.800738 7063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51eaef3e1921403d8174c6803d1a3ceb, State: Initialized, Role: FOLLOWER
I20260926 13:04:10.800906 7063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } }
I20260926 13:04:10.800992 7063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:10.801048 7063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:10.801098 7063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:10.801856 7063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } }
I20260926 13:04:10.801971 7063 leader_election.cc:302] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51eaef3e1921403d8174c6803d1a3ceb; no voters:
I20260926 13:04:10.802124 7063 leader_election.cc:288] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:10.802196 7068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:10.802323 7068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [term 1 LEADER]: Becoming Leader. State: Replica: 51eaef3e1921403d8174c6803d1a3ceb, State: Running, Role: LEADER
I20260926 13:04:10.802383 7063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:10.802413 7068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } }
I20260926 13:04:10.802783 7070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51eaef3e1921403d8174c6803d1a3ceb. Latest consensus state: current_term: 1 leader_uuid: "51eaef3e1921403d8174c6803d1a3ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } } }
I20260926 13:04:10.802786 7069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51eaef3e1921403d8174c6803d1a3ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51eaef3e1921403d8174c6803d1a3ceb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33993 } } }
I20260926 13:04:10.802964 7070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:10.803002 7069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:10.803407 7077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:10.803898 7077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:10.805322 7077 catalog_manager.cc:1347] Generated new cluster ID: 2fe129fb4b5f40a596f6da3c3d41bba2
I20260926 13:04:10.805379 7077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:10.811156 7077 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:10.811307 7077 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:10.817474 7077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: Generated new TSK 0
I20260926 13:04:10.817644 7077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:10.824455 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--builtin_ntp_servers=127.2.86.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:04:10.904000 7087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:10.904186 7087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:10.904207 7087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:10.905404 7087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:10.905440 7087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:10.905465 7087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:10.906776 7087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:10.907022 7087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:10.907305 7087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:10.912708 7093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.912721 7095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:10.912937 7092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:10.913130 7087 server_base.cc:1034] running on GCE node
I20260926 13:04:10.913311 7087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:10.913602 7087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:10.914753 7087 hybrid_clock.cc:648] HybridClock initialized: now 1790427850914733 us; error 36 us; skew 500 ppm
I20260926 13:04:10.915997 7087 webserver.cc:460] Webserver started at http://127.2.86.1:42921/ using document root <none> and password file <none>
I20260926 13:04:10.916255 7087 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:10.916298 7087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:10.916424 7087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:10.917495 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "69224dfcaa0644f5964b14a84c2ab44d"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "46610ab11d56ab0691958807a966d1e2"
server_key_iv: "2d35214541967db941b988e0f9a2ee43"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.917825 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "69224dfcaa0644f5964b14a84c2ab44d"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "46610ab11d56ab0691958807a966d1e2"
server_key_iv: "2d35214541967db941b988e0f9a2ee43"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.919163 7087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:10.919881 7101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.920060 7087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:10.920171 7087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "69224dfcaa0644f5964b14a84c2ab44d"
format_stamp: "Formatted at 2026-09-26 13:04:10 on dist-test-slave-j4lq"
server_key: "46610ab11d56ab0691958807a966d1e2"
server_key_iv: "2d35214541967db941b988e0f9a2ee43"
server_key_version: "encryptionkey@0"
I20260926 13:04:10.920248 7087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:10.923581 7087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:10.923841 7087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:10.923987 7087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:10.924185 7087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:10.924489 7087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:10.924542 7087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.924593 7087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:10.924619 7087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:10.930205 7087 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:32811
I20260926 13:04:10.930251 7214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:32811 every 8 connection(s)
I20260926 13:04:10.930543 7087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:10.933583 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.931247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57029 (local address 127.2.86.62:33993)
0926 13:04:10.931309 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:10.931310 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:10.931543 (+ 233us) server_negotiation.cc:407] Connection header received
0926 13:04:10.932043 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:10.932044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:10.932051 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:10.932107 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:10.933342 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:10.933344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:10.933344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:10.933349 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:10.933363 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:10.933367 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:10.933418 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:10.933499 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 13:04:10.933513 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:04:10.933766 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:10.931239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33993 (local address 127.2.86.1:57029)
0926 13:04:10.931427 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:10.931460 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:10.931931 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:10.932176 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:10.932180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:10.932190 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:10.933195 (+ 1005us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:10.933196 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:10.933222 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:10.933248 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:10.933416 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:10.933420 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:10.933424 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:10.933499 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 13:04:10.933558 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260926 13:04:10.934347 7215 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33993
I20260926 13:04:10.934451 7215 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:10.934609 7215 heartbeater.cc:511] Master 127.2.86.62:33993 requested a full tablet report, sending...
I20260926 13:04:10.935106 7027 ts_manager.cc:194] Registered new tserver with Master: 69224dfcaa0644f5964b14a84c2ab44d (127.2.86.1:32811)
I20260926 13:04:10.939131 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7087
I20260926 13:04:10.939235 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:10.939431 2392 external_mini_cluster.cc:1428] Setting key 6c4b209b377c812cbbbfa22d834cfbc8
I20260926 13:04:10.940321 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--builtin_ntp_servers=127.2.86.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:04:11.019405 7218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:11.019593 7218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:11.019613 7218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:11.020716 7218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:11.020753 7218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:11.020778 7218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:11.022233 7218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:11.022461 7218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:11.022794 7218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:11.027832 7224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.027822 7223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.027825 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:11.028070 7218 server_base.cc:1034] running on GCE node
I20260926 13:04:11.028251 7218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:11.028472 7218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:11.029632 7218 hybrid_clock.cc:648] HybridClock initialized: now 1790427851029625 us; error 40 us; skew 500 ppm
I20260926 13:04:11.030851 7218 webserver.cc:460] Webserver started at http://127.2.86.2:37651/ using document root <none> and password file <none>
I20260926 13:04:11.031100 7218 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:11.031172 7218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:11.031304 7218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:11.032280 7218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "43fdc07dd9a64b7a887771b06111b364"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "41cee3a39a0d6a34f380b4e70ef6513f"
server_key_iv: "1b465a26991b892eebcdbd213784efed"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.032634 7218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "43fdc07dd9a64b7a887771b06111b364"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "41cee3a39a0d6a34f380b4e70ef6513f"
server_key_iv: "1b465a26991b892eebcdbd213784efed"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.033963 7218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:11.034682 7232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.034839 7218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:11.034937 7218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "43fdc07dd9a64b7a887771b06111b364"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "41cee3a39a0d6a34f380b4e70ef6513f"
server_key_iv: "1b465a26991b892eebcdbd213784efed"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.035019 7218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:11.038256 7218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:11.038501 7218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:11.038636 7218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:11.038839 7218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:11.039155 7218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:11.039213 7218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.039250 7218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:11.039278 7218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.044914 7218 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:37933
I20260926 13:04:11.044963 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:37933 every 8 connection(s)
I20260926 13:04:11.045233 7218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:11.047819 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.045882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:37323 (local address 127.2.86.62:33993)
0926 13:04:11.045945 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.045947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.046125 (+ 178us) server_negotiation.cc:407] Connection header received
0926 13:04:11.046527 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.046528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.046533 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.046588 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.047658 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.047659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.047659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.047662 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.047671 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.047674 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.047711 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.047759 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.047769 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:04:11.048000 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.045849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33993 (local address 127.2.86.2:37323)
0926 13:04:11.046007 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.046042 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.046452 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.046632 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.046635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.046642 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.047541 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.047542 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.047566 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.047590 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.047711 (+ 121us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.047713 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.047715 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.047759 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.047801 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260926 13:04:11.048460 7346 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33993
I20260926 13:04:11.048532 7346 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:11.048679 7346 heartbeater.cc:511] Master 127.2.86.62:33993 requested a full tablet report, sending...
I20260926 13:04:11.049052 7027 ts_manager.cc:194] Registered new tserver with Master: 43fdc07dd9a64b7a887771b06111b364 (127.2.86.2:37933)
I20260926 13:04:11.054862 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7218
I20260926 13:04:11.054970 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:11.055138 2392 external_mini_cluster.cc:1428] Setting key 6be4c989b027401ed9aa9ecd24dc7b15
I20260926 13:04:11.055967 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--builtin_ntp_servers=127.2.86.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 13:04:11.133247 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:11.133412 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:11.133433 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:11.134516 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:11.134552 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:11.134598 7349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:11.135824 7349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:11.136050 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:11.136332 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:11.141068 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.141074 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.141103 7357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:11.141274 7349 server_base.cc:1034] running on GCE node
I20260926 13:04:11.141441 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:11.141623 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:11.142755 7349 hybrid_clock.cc:648] HybridClock initialized: now 1790427851142731 us; error 39 us; skew 500 ppm
I20260926 13:04:11.143873 7349 webserver.cc:460] Webserver started at http://127.2.86.3:41775/ using document root <none> and password file <none>
I20260926 13:04:11.144107 7349 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:11.144150 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:11.144311 7349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:11.145388 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "459a9c3848e3461aba5cb41e16943fe3"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "dd72aaf0157249f361f17a9ae1780e43"
server_key_iv: "4cda4162495f926b26eaa5833eb7945c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.145723 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "459a9c3848e3461aba5cb41e16943fe3"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "dd72aaf0157249f361f17a9ae1780e43"
server_key_iv: "4cda4162495f926b26eaa5833eb7945c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.146961 7349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:11.147634 7363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.147819 7349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:11.147886 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "459a9c3848e3461aba5cb41e16943fe3"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "dd72aaf0157249f361f17a9ae1780e43"
server_key_iv: "4cda4162495f926b26eaa5833eb7945c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.147977 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:11.151088 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:11.151321 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:11.151459 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:11.151651 7349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:11.151928 7349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:11.151958 7349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.152009 7349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:11.152035 7349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.157387 7349 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:34983
I20260926 13:04:11.157445 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:34983 every 8 connection(s)
I20260926 13:04:11.157747 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:11.160142 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7349
I20260926 13:04:11.160269 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:11.160449 2392 external_mini_cluster.cc:1428] Setting key f75880da3f5863d94bdb50b0cb522469
I20260926 13:04:11.160612 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.158424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38153 (local address 127.2.86.62:33993)
0926 13:04:11.158495 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.158496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.158716 (+ 220us) server_negotiation.cc:407] Connection header received
0926 13:04:11.159108 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.159109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.159113 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.159169 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.160320 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.160321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.160321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.160327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.160343 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.160347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.160443 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.160529 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.160544 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:04:11.160786 7479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.158412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33993 (local address 127.2.86.3:38153)
0926 13:04:11.158593 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.158629 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.159031 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.159213 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.159216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.159223 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.160145 (+ 922us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.160145 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.160171 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.160194 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.160445 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.160450 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.160454 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.160531 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.160587 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260926 13:04:11.161278 7477 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33993
I20260926 13:04:11.161363 7477 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:11.161496 7477 heartbeater.cc:511] Master 127.2.86.62:33993 requested a full tablet report, sending...
I20260926 13:04:11.161811 7027 ts_manager.cc:194] Registered new tserver with Master: 459a9c3848e3461aba5cb41e16943fe3 (127.2.86.3:34983)
I20260926 13:04:11.172335 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:04:11.174538 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.173657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.2.86.62:33993)
0926 13:04:11.173711 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.173713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.173805 (+ 92us) server_negotiation.cc:407] Connection header received
0926 13:04:11.173843 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.173843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.173847 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.173892 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.174374 (+ 482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.174375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.174375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.174378 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.174389 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.174392 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.174434 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.174482 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.174493 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:04:11.175931 7027 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:11.177539 7027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:11.181291 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:37933 (local address 127.0.0.1:48596)
0926 13:04:11.180024 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.180033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.180082 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.180244 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.180249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.180256 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.180709 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.180709 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.180722 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.180738 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.181124 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.181136 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.181139 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.181206 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.181220 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260926 13:04:11.181417 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.2.86.2:37933)
0926 13:04:11.180018 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.180022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.180105 (+ 83us) server_negotiation.cc:407] Connection header received
0926 13:04:11.180135 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.180137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.180147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.180237 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.180989 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.181000 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.181000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.181006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.181041 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.181070 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.181133 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.181226 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.181243 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":83,"threads_started":1}
I20260926 13:04:11.181638 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34983 (local address 127.0.0.1:53814)
0926 13:04:11.179859 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.179896 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.179948 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.180449 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.180450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.180451 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.181178 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.181178 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.181185 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.181196 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.181493 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.181494 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.181495 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.181551 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.181565 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":90,"threads_started":1}
I20260926 13:04:11.181872 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.2.86.3:34983)
0926 13:04:11.179921 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.179925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.179938 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:04:11.180033 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.180040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.180052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.180121 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.181270 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.181276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.181276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.181280 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.181309 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.181331 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.181401 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.181777 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.181794 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260926 13:04:11.182106 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:32811 (local address 127.0.0.1:44550)
0926 13:04:11.180199 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.180209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.180247 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.180995 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.180996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.180998 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.181734 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.181734 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.181740 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.181752 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.181985 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.181986 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.181987 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.182032 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.182044 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":67,"threads_started":1}
I20260926 13:04:11.182124 7280 tablet_service.cc:1467] Processing CreateTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 (DEFAULT_TABLE table=test-table [id=f369f17b18b44ab28c58b9ca4df9f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:11.182294 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.179996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.2.86.1:32811)
0926 13:04:11.180576 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.180581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.180590 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:11.180615 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.180617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.180625 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.180675 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.181835 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.181841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.181842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.181846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.181877 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.181902 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.181958 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.182199 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.182216 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":110,"threads_started":1}
I20260926 13:04:11.182453 7280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79dc1f6eee2746cb8105c62adadbd737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:11.182453 7411 tablet_service.cc:1467] Processing CreateTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 (DEFAULT_TABLE table=test-table [id=f369f17b18b44ab28c58b9ca4df9f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:11.182749 7411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79dc1f6eee2746cb8105c62adadbd737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:11.182852 7149 tablet_service.cc:1467] Processing CreateTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 (DEFAULT_TABLE table=test-table [id=f369f17b18b44ab28c58b9ca4df9f606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:11.183071 7149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79dc1f6eee2746cb8105c62adadbd737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:11.185416 7491 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Bootstrap starting.
I20260926 13:04:11.185850 7493 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Bootstrap starting.
I20260926 13:04:11.186362 7491 tablet_bootstrap.cc:654] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:11.186457 7493 tablet_bootstrap.cc:654] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:11.186713 7491 log.cc:824] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:11.186769 7493 log.cc:824] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:11.187055 7492 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Bootstrap starting.
I20260926 13:04:11.187494 7491 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: No bootstrap required, opened a new log
I20260926 13:04:11.187520 7493 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: No bootstrap required, opened a new log
I20260926 13:04:11.187578 7493 ts_tablet_manager.cc:1397] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:11.187579 7491 ts_tablet_manager.cc:1397] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:11.187657 7492 tablet_bootstrap.cc:654] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:11.187958 7492 log.cc:824] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:11.188771 7492 tablet_bootstrap.cc:492] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: No bootstrap required, opened a new log
I20260926 13:04:11.188886 7492 ts_tablet_manager.cc:1397] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:11.190054 7491 raft_consensus.cc:348] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.190196 7491 raft_consensus.cc:374] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:11.190235 7491 raft_consensus.cc:729] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69224dfcaa0644f5964b14a84c2ab44d, State: Initialized, Role: FOLLOWER
I20260926 13:04:11.190232 7493 raft_consensus.cc:348] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.190385 7493 raft_consensus.cc:374] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:11.190338 7491 consensus_queue.cc:260] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.190428 7493 raft_consensus.cc:729] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43fdc07dd9a64b7a887771b06111b364, State: Initialized, Role: FOLLOWER
I20260926 13:04:11.190546 7493 consensus_queue.cc:260] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.190629 7491 ts_tablet_manager.cc:1428] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:04:11.190634 7215 heartbeater.cc:503] Master 127.2.86.62:33993 was elected leader, sending a full tablet report...
I20260926 13:04:11.190794 7493 ts_tablet_manager.cc:1428] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 13:04:11.190918 7346 heartbeater.cc:503] Master 127.2.86.62:33993 was elected leader, sending a full tablet report...
I20260926 13:04:11.191524 7492 raft_consensus.cc:348] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.191639 7492 raft_consensus.cc:374] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:11.191659 7492 raft_consensus.cc:729] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 459a9c3848e3461aba5cb41e16943fe3, State: Initialized, Role: FOLLOWER
I20260926 13:04:11.191753 7492 consensus_queue.cc:260] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.191965 7492 ts_tablet_manager.cc:1428] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:04:11.192034 7477 heartbeater.cc:503] Master 127.2.86.62:33993 was elected leader, sending a full tablet report...
W20260926 13:04:11.295794 7347 tablet.cc:2369] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:04:11.408254 7478 tablet.cc:2369] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:04:11.431160 7216 tablet.cc:2369] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:04:11.434171 7498 raft_consensus.cc:482] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:11.434312 7498 raft_consensus.cc:504] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.434682 7498 leader_election.cc:288] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69224dfcaa0644f5964b14a84c2ab44d (127.2.86.1:32811), 459a9c3848e3461aba5cb41e16943fe3 (127.2.86.3:34983)
I20260926 13:04:11.436067 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.434969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51749 (local address 127.2.86.3:34983)
0926 13:04:11.435024 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.435026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.435035 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:11.435064 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.435065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.435069 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.435156 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.435786 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.435788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.435788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.435793 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.435817 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.435823 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.435894 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.435992 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.436007 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:11.436066 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.434874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34983 (local address 127.2.86.2:51749)
0926 13:04:11.434941 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.434966 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.435004 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.435164 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.435165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.435167 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.435664 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.435664 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.435672 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.435686 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.435900 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.435901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.435902 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.435984 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.435999 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 13:04:11.436200 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.434933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:54775 (local address 127.2.86.1:32811)
0926 13:04:11.434981 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.434983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.435154 (+ 171us) server_negotiation.cc:407] Connection header received
0926 13:04:11.435215 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.435216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.435222 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.435273 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.435959 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.435960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.435961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.435965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.435979 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.435983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.436054 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.436126 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.436139 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:04:11.436211 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.434883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:32811 (local address 127.2.86.2:54775)
0926 13:04:11.435076 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.435083 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.435156 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.435276 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.435277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.435279 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:11.435851 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:11.435852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:11.435870 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:11.435884 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:11.436052 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:11.436053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:11.436053 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 13:04:11.436105 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:04:11.436129 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":114,"threads_started":1}
I20260926 13:04:11.436424 7431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79dc1f6eee2746cb8105c62adadbd737" candidate_uuid: "43fdc07dd9a64b7a887771b06111b364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "459a9c3848e3461aba5cb41e16943fe3" is_pre_election: true
I20260926 13:04:11.436501 7169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79dc1f6eee2746cb8105c62adadbd737" candidate_uuid: "43fdc07dd9a64b7a887771b06111b364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69224dfcaa0644f5964b14a84c2ab44d" is_pre_election: true
I20260926 13:04:11.436575 7431 raft_consensus.cc:2454] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43fdc07dd9a64b7a887771b06111b364 in term 0.
I20260926 13:04:11.436693 7169 raft_consensus.cc:2454] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43fdc07dd9a64b7a887771b06111b364 in term 0.
I20260926 13:04:11.436857 7233 leader_election.cc:302] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43fdc07dd9a64b7a887771b06111b364, 459a9c3848e3461aba5cb41e16943fe3; no voters:
I20260926 13:04:11.437034 7498 raft_consensus.cc:2781] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:11.437101 7498 raft_consensus.cc:482] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:11.437134 7498 raft_consensus.cc:3037] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:11.437822 7498 raft_consensus.cc:504] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.437978 7498 leader_election.cc:288] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [CANDIDATE]: Term 1 election: Requested vote from peers 69224dfcaa0644f5964b14a84c2ab44d (127.2.86.1:32811), 459a9c3848e3461aba5cb41e16943fe3 (127.2.86.3:34983)
I20260926 13:04:11.438114 7431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79dc1f6eee2746cb8105c62adadbd737" candidate_uuid: "43fdc07dd9a64b7a887771b06111b364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "459a9c3848e3461aba5cb41e16943fe3"
I20260926 13:04:11.438143 7169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79dc1f6eee2746cb8105c62adadbd737" candidate_uuid: "43fdc07dd9a64b7a887771b06111b364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69224dfcaa0644f5964b14a84c2ab44d"
I20260926 13:04:11.438205 7169 raft_consensus.cc:3037] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:11.438199 7431 raft_consensus.cc:3037] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:11.439079 7169 raft_consensus.cc:2454] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43fdc07dd9a64b7a887771b06111b364 in term 1.
I20260926 13:04:11.439080 7431 raft_consensus.cc:2454] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43fdc07dd9a64b7a887771b06111b364 in term 1.
I20260926 13:04:11.439247 7233 leader_election.cc:302] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43fdc07dd9a64b7a887771b06111b364, 459a9c3848e3461aba5cb41e16943fe3; no voters:
I20260926 13:04:11.439343 7498 raft_consensus.cc:2781] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:11.439497 7498 raft_consensus.cc:686] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 1 LEADER]: Becoming Leader. State: Replica: 43fdc07dd9a64b7a887771b06111b364, State: Running, Role: LEADER
I20260926 13:04:11.439592 7498 consensus_queue.cc:237] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } }
I20260926 13:04:11.440339 7026 catalog_manager.cc:5495] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43fdc07dd9a64b7a887771b06111b364 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "43fdc07dd9a64b7a887771b06111b364" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43fdc07dd9a64b7a887771b06111b364" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 37933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:11.465998 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.464965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.2.86.2:37933)
0926 13:04:11.465022 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.465024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.465032 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:11.465078 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.465079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.465091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.465157 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.465771 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.465772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.465773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.465778 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.465792 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.465797 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.465849 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.465908 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.465921 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:04:11.467804 7431 raft_consensus.cc:1261] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 1 FOLLOWER]: Refusing update from remote peer 43fdc07dd9a64b7a887771b06111b364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:11.467804 7169 raft_consensus.cc:1261] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 1 FOLLOWER]: Refusing update from remote peer 43fdc07dd9a64b7a887771b06111b364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:11.468200 7501 consensus_queue.cc:1035] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69224dfcaa0644f5964b14a84c2ab44d" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 32811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:11.468329 7498 consensus_queue.cc:1035] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "459a9c3848e3461aba5cb41e16943fe3" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:11.470765 7506 mvcc.cc:205] Tried to move back new op lower bound from 7333592479609241600 to 7333592479497039872. Current Snapshot: MvccSnapshot[applied={T|T < 7333592479609241600}]
I20260926 13:04:11.471184 7505 mvcc.cc:205] Tried to move back new op lower bound from 7333592479609241600 to 7333592479497039872. Current Snapshot: MvccSnapshot[applied={T|T < 7333592479609241600}]
I20260926 13:04:11.472625 7027 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:11.472704 7027 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:11.473476 7027 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51eaef3e1921403d8174c6803d1a3ceb: Sending DeleteTablet for 3 replicas of tablet 79dc1f6eee2746cb8105c62adadbd737
I20260926 13:04:11.473773 7280 tablet_service.cc:1514] Processing DeleteTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:11 UTC) from {username='slave'} at 127.0.0.1:48596
I20260926 13:04:11.473829 7149 tablet_service.cc:1514] Processing DeleteTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:11 UTC) from {username='slave'} at 127.0.0.1:44550
I20260926 13:04:11.473886 7411 tablet_service.cc:1514] Processing DeleteTablet for tablet 79dc1f6eee2746cb8105c62adadbd737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:11 UTC) from {username='slave'} at 127.0.0.1:53814
I20260926 13:04:11.474009 7513 tablet_replica.cc:330] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: stopping tablet replica
I20260926 13:04:11.474064 7514 tablet_replica.cc:330] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d: stopping tablet replica
I20260926 13:04:11.474102 7513 raft_consensus.cc:2229] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:11.474138 7514 raft_consensus.cc:2229] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:11.474216 7513 raft_consensus.cc:2258] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:11.474234 7514 raft_consensus.cc:2258] T 79dc1f6eee2746cb8105c62adadbd737 P 69224dfcaa0644f5964b14a84c2ab44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:11.474459 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7087
I20260926 13:04:11.474642 7513 ts_tablet_manager.cc:1905] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:11.476336 7515 tablet_replica.cc:330] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: stopping tablet replica
I20260926 13:04:11.476495 7515 raft_consensus.cc:2229] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:11.476644 7515 raft_consensus.cc:2258] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:11.477131 7515 ts_tablet_manager.cc:1905] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:11.477392 7513 ts_tablet_manager.cc:1918] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:11.477535 7513 log.cc:1196] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/79dc1f6eee2746cb8105c62adadbd737
I20260926 13:04:11.477804 7513 ts_tablet_manager.cc:1939] T 79dc1f6eee2746cb8105c62adadbd737 P 43fdc07dd9a64b7a887771b06111b364: Deleting consensus metadata
I20260926 13:04:11.478174 7014 catalog_manager.cc:4841] TS 43fdc07dd9a64b7a887771b06111b364 (127.2.86.2:37933): tablet 79dc1f6eee2746cb8105c62adadbd737 (table test-table [id=f369f17b18b44ab28c58b9ca4df9f606]) successfully deleted
I20260926 13:04:11.479044 7515 ts_tablet_manager.cc:1918] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:11.479182 7515 log.cc:1196] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/79dc1f6eee2746cb8105c62adadbd737
I20260926 13:04:11.479506 7515 ts_tablet_manager.cc:1939] T 79dc1f6eee2746cb8105c62adadbd737 P 459a9c3848e3461aba5cb41e16943fe3: Deleting consensus metadata
I20260926 13:04:11.479931 7011 catalog_manager.cc:4841] TS 459a9c3848e3461aba5cb41e16943fe3 (127.2.86.3:34983): tablet 79dc1f6eee2746cb8105c62adadbd737 (table test-table [id=f369f17b18b44ab28c58b9ca4df9f606]) successfully deleted
W20260926 13:04:11.480309 7014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:32811 (error 108)
I20260926 13:04:11.480546 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7218
I20260926 13:04:11.481233 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.480685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:32811 (local address 127.0.0.1:56320)
0926 13:04:11.481175 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.481208 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464}
W20260926 13:04:11.481365 7014 catalog_manager.cc:4584] TS 69224dfcaa0644f5964b14a84c2ab44d (127.2.86.1:32811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79dc1f6eee2746cb8105c62adadbd737: Network error: Client connection negotiation failed: client connection to 127.2.86.1:32811: connect: Connection refused (error 111)
I20260926 13:04:11.485829 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7349
I20260926 13:04:11.490064 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 6996
2026-09-26T13:04:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (822 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 13:04:11.509294 2392 mini_oidc.cc:152] Starting JWKS server
I20260926 13:04:11.510317 2392 webserver.cc:460] Webserver started at https://127.0.0.1:46713/ using document root <none> and password file <none>
I20260926 13:04:11.510423 2392 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:04:11.510617 2392 webserver.cc:460] Webserver started at http://127.0.0.1:41559/ using document root <none> and password file <none>
2026-09-26T13:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:11Z Disabled control of system clock
I20260926 13:04:11.517968 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41933
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:45789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:41933
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46713/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/testchainca.cert with env {}
W20260926 13:04:11.596123 7522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:11.596318 7522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:11.596340 7522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:11.597512 7522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:11.597551 7522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:11.597564 7522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:11.597577 7522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:11.597589 7522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:04:11.597602 7522 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:04:11.597613 7522 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46713/jwks.json
I20260926 13:04:11.599051 7522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:41933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46713/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:11.599229 7522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:11.599380 7522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:11.604013 7527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.604070 7530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.604013 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:11.604190 7522 server_base.cc:1034] running on GCE node
I20260926 13:04:11.604403 7522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:11.604629 7522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:11.605746 7522 hybrid_clock.cc:648] HybridClock initialized: now 1790427851605735 us; error 27 us; skew 500 ppm
I20260926 13:04:11.606933 7522 webserver.cc:460] Webserver started at http://127.2.86.62:45983/ using document root <none> and password file <none>
I20260926 13:04:11.607169 7522 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:11.607214 7522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:11.607354 7522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:11.608345 7522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "f6447770764e4a67b421df7508563026"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "b4c6153498b74362c44d266f8545c018"
server_key_iv: "5f45c05f0dda86534ee97e164129ff9f"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.608697 7522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "f6447770764e4a67b421df7508563026"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "b4c6153498b74362c44d266f8545c018"
server_key_iv: "5f45c05f0dda86534ee97e164129ff9f"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.610019 7522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:11.610767 7536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.610916 7522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:11.611016 7522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "f6447770764e4a67b421df7508563026"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "b4c6153498b74362c44d266f8545c018"
server_key_iv: "5f45c05f0dda86534ee97e164129ff9f"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.611094 7522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:11.617614 7522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:11.617916 7522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:11.618068 7522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:11.621874 7522 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:41933
I20260926 13:04:11.621902 7588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:41933 every 8 connection(s)
I20260926 13:04:11.622198 7522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:11.623047 7589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:11.625227 7589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026: Bootstrap starting.
I20260926 13:04:11.625728 7589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:11.625958 7589 log.cc:824] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:11.626533 7589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026: No bootstrap required, opened a new log
I20260926 13:04:11.627990 7589 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:04:11.628108 7589 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:11.628129 7589 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6447770764e4a67b421df7508563026, State: Initialized, Role: FOLLOWER
I20260926 13:04:11.628250 7589 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:04:11.628324 7589 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:11.628345 7589 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:11.628369 7589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:11.628819 7589 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:04:11.628973 7589 leader_election.cc:302] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6447770764e4a67b421df7508563026; no voters:
I20260926 13:04:11.629096 7589 leader_election.cc:288] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:11.629179 7592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:11.629330 7592 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [term 1 LEADER]: Becoming Leader. State: Replica: f6447770764e4a67b421df7508563026, State: Running, Role: LEADER
I20260926 13:04:11.629364 7589 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:11.629423 7592 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:04:11.629926 7593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6447770764e4a67b421df7508563026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } } }
I20260926 13:04:11.630111 7594 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6447770764e4a67b421df7508563026. Latest consensus state: current_term: 1 leader_uuid: "f6447770764e4a67b421df7508563026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6447770764e4a67b421df7508563026" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } } }
I20260926 13:04:11.630184 7593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:11.630208 7594 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:11.630515 7603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:11.630962 7603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:11.631639 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7522
I20260926 13:04:11.631755 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:11.631935 2392 external_mini_cluster.cc:1428] Setting key 9eec3f1eb29d6948ee670c45af6fea32
I20260926 13:04:11.632519 7603 catalog_manager.cc:1347] Generated new cluster ID: 97b9a24978bd468fa7b3c22fcf1b4377
I20260926 13:04:11.632566 7603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:11.642088 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.632327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.2.86.62:41933)
0926 13:04:11.632498 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.632507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.637128 (+ 4621us) server_negotiation.cc:407] Connection header received
0926 13:04:11.637242 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.637249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.637320 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.637458 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.637667 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.638961 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.640067 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.640636 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.640739 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.641448 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.641452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.641452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.641632 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.641668 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.641680 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.641742 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.641825 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.641891 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260926 13:04:11.642863 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:04:11.659652 7603 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:11.660332 7603 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:11.664826 7603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6447770764e4a67b421df7508563026: Generated new TSK 0
I20260926 13:04:11.665012 7603 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 13:04:11.691030 7619 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 13:04:11.691191 7619 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 13:04:11.691350 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.676535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.2.86.62:41933)
0926 13:04:11.676577 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.676579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.676704 (+ 125us) server_negotiation.cc:407] Connection header received
0926 13:04:11.676778 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.676779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.676790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.676884 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:04:11.677053 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.677762 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.678485 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.678790 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.678854 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.679055 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:04:11.691160 (+ 12105us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 13:04:11.691288 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 13:04:11.691406 7612 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 13:04:11.691720 7618 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:11.676458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:41933 (local address 127.0.0.1:58822)
0926 13:04:11.676637 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.676646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.676701 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.676930 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.676931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.676933 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:04:11.676988 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:11.676989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.677851 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.677852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:11.678411 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:11.678412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.678880 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.678883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:11.678933 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.678950 (+ 17us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:04:11.691323 (+ 12373us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 13:04:11.691446 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.86.62:41933: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.86.62:41933: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 13:04:11.692322 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7522
2026-09-26T13:04:11Z chronyd exiting
I20260926 13:04:11.728181 2392 test_util.cc:175] -----------------------------------------------
I20260926 13:04:11.728276 2392 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790427826505950-2392-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 13:04:11.729173 2392 mini_oidc.cc:152] Starting JWKS server
I20260926 13:04:11.729851 2392 webserver.cc:460] Webserver started at https://127.0.0.1:38701/ using document root <none> and password file <none>
I20260926 13:04:11.729931 2392 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:04:11.730098 2392 webserver.cc:460] Webserver started at http://127.0.0.1:35619/ using document root <none> and password file <none>
2026-09-26T13:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:11Z Disabled control of system clock
I20260926 13:04:11.737648 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41175
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:41175
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38701/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/testchainca.cert with env {}
W20260926 13:04:11.816433 7625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:11.816625 7625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:11.816646 7625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:11.817885 7625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:11.817927 7625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:11.817940 7625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:11.817953 7625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:11.817998 7625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:04:11.818023 7625 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:04:11.818048 7625 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38701/jwks.json
I20260926 13:04:11.819578 7625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:41175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38701/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:11.819828 7625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:11.820019 7625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:11.824612 7633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.824640 7631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:11.824694 7630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:11.824780 7625 server_base.cc:1034] running on GCE node
I20260926 13:04:11.825018 7625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:11.825322 7625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:11.826462 7625 hybrid_clock.cc:648] HybridClock initialized: now 1790427851826440 us; error 36 us; skew 500 ppm
I20260926 13:04:11.827728 7625 webserver.cc:460] Webserver started at http://127.2.86.62:45763/ using document root <none> and password file <none>
I20260926 13:04:11.827962 7625 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:11.828008 7625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:11.828162 7625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:11.829252 7625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "2a6513e77a8742d0a44d68077ac47aec"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "f45c749bcf9f2462658617297cc00a83"
server_key_iv: "f9efbd6de7581642fb25a3357b421d2c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.829607 7625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "2a6513e77a8742d0a44d68077ac47aec"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "f45c749bcf9f2462658617297cc00a83"
server_key_iv: "f9efbd6de7581642fb25a3357b421d2c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.830931 7625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:11.831643 7639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:11.831804 7625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:11.831912 7625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "2a6513e77a8742d0a44d68077ac47aec"
format_stamp: "Formatted at 2026-09-26 13:04:11 on dist-test-slave-j4lq"
server_key: "f45c749bcf9f2462658617297cc00a83"
server_key_iv: "f9efbd6de7581642fb25a3357b421d2c"
server_key_version: "encryptionkey@0"
I20260926 13:04:11.831982 7625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:11.840445 7625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:11.840710 7625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:11.840921 7625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:11.844533 7625 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:41175
I20260926 13:04:11.844561 7691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:41175 every 8 connection(s)
I20260926 13:04:11.844915 7625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:11.845764 7692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:11.847862 7692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec: Bootstrap starting.
I20260926 13:04:11.848390 7692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:11.848676 7692 log.cc:824] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:11.849331 7692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec: No bootstrap required, opened a new log
I20260926 13:04:11.850821 7692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } }
I20260926 13:04:11.850942 7692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:11.850972 7692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6513e77a8742d0a44d68077ac47aec, State: Initialized, Role: FOLLOWER
I20260926 13:04:11.851094 7692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } }
I20260926 13:04:11.851166 7692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:11.851186 7692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:11.851245 7692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:11.851595 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7625
I20260926 13:04:11.851706 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:11.851898 2392 external_mini_cluster.cc:1428] Setting key de765eb1e5b50e484fac3d0356ea20a9
I20260926 13:04:11.852010 7692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } }
I20260926 13:04:11.852131 7692 leader_election.cc:302] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a6513e77a8742d0a44d68077ac47aec; no voters:
I20260926 13:04:11.852283 7692 leader_election.cc:288] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:11.852432 7697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:11.852519 7692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:11.852633 7697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [term 1 LEADER]: Becoming Leader. State: Replica: 2a6513e77a8742d0a44d68077ac47aec, State: Running, Role: LEADER
I20260926 13:04:11.852766 7697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } }
I20260926 13:04:11.853173 7695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a6513e77a8742d0a44d68077ac47aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } } }
I20260926 13:04:11.853262 7695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:11.853328 7699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a6513e77a8742d0a44d68077ac47aec. Latest consensus state: current_term: 1 leader_uuid: "2a6513e77a8742d0a44d68077ac47aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6513e77a8742d0a44d68077ac47aec" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41175 } } }
I20260926 13:04:11.853384 7699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:11.854012 7706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:11.854681 7706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:11.856142 7706 catalog_manager.cc:1347] Generated new cluster ID: dc601d6df2914680be9849d67bb490a3
I20260926 13:04:11.856192 7706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:11.857825 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.852376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.2.86.62:41175)
0926 13:04:11.852580 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.852588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.852785 (+ 197us) server_negotiation.cc:407] Connection header received
0926 13:04:11.852984 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.852991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.853084 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.853229 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:11.853473 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.854679 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.855546 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.856122 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.856240 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.857149 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:11.857153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:11.857153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:11.857348 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:11.857399 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:11.857412 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:11.857472 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:11.857568 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 13:04:11.857637 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260926 13:04:11.858659 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:04:11.862771 7706 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:11.863251 7706 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:11.865978 7706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a6513e77a8742d0a44d68077ac47aec: Generated new TSK 0
I20260926 13:04:11.866145 7706 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 13:04:11.898128 7722 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 13:04:11.898253 7722 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 13:04:11.898490 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:11.875122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.2.86.62:41175)
0926 13:04:11.875187 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:11.875189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:11.875284 (+ 95us) server_negotiation.cc:407] Connection header received
0926 13:04:11.875378 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:11.875379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:11.875392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:11.875485 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:04:11.875681 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.876441 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.877675 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.878042 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.878081 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.878343 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:04:11.898228 (+ 19885us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 13:04:11.898404 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48592: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 13:04:11.898553 7698 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48592: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 13:04:11.898577 7721 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:11.875103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:41175 (local address 127.0.0.1:48592)
0926 13:04:11.875233 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:11.875242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:11.875296 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:11.875533 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:11.875534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:11.875536 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:04:11.875591 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:11.875593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.876557 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.876558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:11.877570 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:11.877572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:11.878139 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:11.878142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:11.878195 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:11.878200 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:04:11.898375 (+ 20175us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 13:04:11.898478 (+ 103us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.86.62:41175: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38701/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260926 13:04:11.898921 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7625
2026-09-26T13:04:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (206 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 13:04:11.936066 2392 mini_oidc.cc:152] Starting JWKS server
I20260926 13:04:11.936759 2392 webserver.cc:460] Webserver started at https://127.0.0.1:41429/ using document root <none> and password file <none>
I20260926 13:04:11.936801 2392 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 13:04:11.937009 2392 webserver.cc:460] Webserver started at http://127.0.0.1:36105/ using document root <none> and password file <none>
2026-09-26T13:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:11Z Disabled control of system clock
I20260926 13:04:11.944489 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45885
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:45885
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41429/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 13:04:12.021956 7728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:12.022182 7728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:12.022234 7728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:12.023387 7728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:12.023452 7728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:12.023475 7728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:12.023496 7728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:12.023528 7728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:04:12.023551 7728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:04:12.023572 7728 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41429/jwks.json
I20260926 13:04:12.024927 7728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:45885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41429/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:12.025182 7728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:12.025455 7728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:12.030220 7733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:12.030216 7734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:12.030215 7736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:12.030243 7728 server_base.cc:1034] running on GCE node
I20260926 13:04:12.030767 7728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:12.031028 7728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:12.032166 7728 hybrid_clock.cc:648] HybridClock initialized: now 1790427852032153 us; error 30 us; skew 500 ppm
I20260926 13:04:12.033500 7728 webserver.cc:460] Webserver started at http://127.2.86.62:45017/ using document root <none> and password file <none>
I20260926 13:04:12.033743 7728 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:12.033790 7728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:12.033974 7728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:12.035017 7728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "6e585e8ae8ee42aaac29195151802631"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "979ab747764ba66d79593291aecccc11"
server_key_iv: "2849bb9ad4c1e86bd1f1734e348238a8"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.035359 7728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "6e585e8ae8ee42aaac29195151802631"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "979ab747764ba66d79593291aecccc11"
server_key_iv: "2849bb9ad4c1e86bd1f1734e348238a8"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.036671 7728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:12.037446 7742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:12.037623 7728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:12.037725 7728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "6e585e8ae8ee42aaac29195151802631"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "979ab747764ba66d79593291aecccc11"
server_key_iv: "2849bb9ad4c1e86bd1f1734e348238a8"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.037796 7728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:12.053968 7728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:12.054221 7728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:12.054373 7728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:12.057920 7728 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:45885
I20260926 13:04:12.057983 7794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:45885 every 8 connection(s)
I20260926 13:04:12.058292 7728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:12.058418 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7728
I20260926 13:04:12.058516 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:12.058717 2392 external_mini_cluster.cc:1428] Setting key bdb09d6d5c618c47537318bb84e6e63b
I20260926 13:04:12.059089 7795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:12.062318 7795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631: Bootstrap starting.
I20260926 13:04:12.062942 7795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:12.063251 7795 log.cc:824] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:12.064026 7795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631: No bootstrap required, opened a new log
I20260926 13:04:12.064785 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:12.059492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.2.86.62:45885)
0926 13:04:12.059690 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:12.059699 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:12.059730 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:04:12.060170 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:12.060178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:12.060256 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:12.060399 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:12.060592 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.061919 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.062721 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.063346 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.063474 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.064146 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:12.064150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:12.064151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:12.064311 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:12.064362 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:12.064378 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:12.064455 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:12.064548 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:04:12.064625 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260926 13:04:12.065740 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:04:12.066329 7795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } }
I20260926 13:04:12.066517 7795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:12.066556 7795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e585e8ae8ee42aaac29195151802631, State: Initialized, Role: FOLLOWER
I20260926 13:04:12.066671 7795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } }
I20260926 13:04:12.066756 7795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:12.066785 7795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:12.066829 7795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:12.067283 7795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } }
I20260926 13:04:12.067361 7795 leader_election.cc:302] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e585e8ae8ee42aaac29195151802631; no voters:
I20260926 13:04:12.067471 7795 leader_election.cc:288] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:12.067553 7800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:12.067689 7800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [term 1 LEADER]: Becoming Leader. State: Replica: 6e585e8ae8ee42aaac29195151802631, State: Running, Role: LEADER
I20260926 13:04:12.067721 7795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:12.067775 7800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } }
I20260926 13:04:12.068185 7802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e585e8ae8ee42aaac29195151802631. Latest consensus state: current_term: 1 leader_uuid: "6e585e8ae8ee42aaac29195151802631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } } }
I20260926 13:04:12.068284 7802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:12.068171 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e585e8ae8ee42aaac29195151802631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e585e8ae8ee42aaac29195151802631" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45885 } } }
I20260926 13:04:12.068521 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:12.068495 7809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:12.069116 7809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:12.070556 7809 catalog_manager.cc:1347] Generated new cluster ID: 2afbe511820144158e3e24fac75fc127
I20260926 13:04:12.070607 7809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:12.086133 7809 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:12.086621 7809 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:12.095600 7809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e585e8ae8ee42aaac29195151802631: Generated new TSK 0
I20260926 13:04:12.095786 7809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:12.105376 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:12.102121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.2.86.62:45885)
0926 13:04:12.102182 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:12.102184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:12.102265 (+ 81us) server_negotiation.cc:407] Connection header received
0926 13:04:12.102349 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:12.102350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:12.102362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:12.102454 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:04:12.102581 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.103066 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.103806 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.104025 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.104102 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.104273 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:04:12.105162 (+ 889us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 13:04:12.105302 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53778: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 13:04:12.105430 7798 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53778: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 13:04:12.105683 7824 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:12.102043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45885 (local address 127.0.0.1:53778)
0926 13:04:12.102195 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:12.102205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:12.102283 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:12.102446 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:12.102447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:12.102449 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:04:12.102508 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:12.102510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.103157 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.103158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:12.103728 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:12.103729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.104101 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.104104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:12.104159 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.104164 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:04:12.105268 (+ 1104us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 13:04:12.105355 (+ 87us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.86.62:45885: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260926 13:04:12.105942 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7728
E20260926 13:04:12.115895 7825 webserver.cc:533] Webserver: sslize failed:
E20260926 13:04:12.115948 7825 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T13:04:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (207 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T13:04:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:12Z Disabled control of system clock
I20260926 13:04:12.150344 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46481
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:46481
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/testchainca.cert with env {}
W20260926 13:04:12.229305 7829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:12.229492 7829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:12.229512 7829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:12.230677 7829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:12.230718 7829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:12.230731 7829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:12.230743 7829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:12.230803 7829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 13:04:12.230824 7829 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 13:04:12.230851 7829 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 13:04:12.232299 7829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:46481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:12.232532 7829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:12.232736 7829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:12.237370 7837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:12.237387 7835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:12.237447 7829 server_base.cc:1034] running on GCE node
W20260926 13:04:12.237409 7834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:12.237810 7829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:12.238030 7829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:12.239202 7829 hybrid_clock.cc:648] HybridClock initialized: now 1790427852239185 us; error 30 us; skew 500 ppm
I20260926 13:04:12.240294 7829 webserver.cc:460] Webserver started at http://127.2.86.62:39391/ using document root <none> and password file <none>
I20260926 13:04:12.240513 7829 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:12.240556 7829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:12.240695 7829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:12.241765 7829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "939b841219464bd7b630901961a5144e"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "4409fabdf2f5c52d051773232c9bfbcc"
server_key_iv: "63db12a8a9f296c00cd1b2bbca8441bb"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.242118 7829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "939b841219464bd7b630901961a5144e"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "4409fabdf2f5c52d051773232c9bfbcc"
server_key_iv: "63db12a8a9f296c00cd1b2bbca8441bb"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.243366 7829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:12.244088 7843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:12.244241 7829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:12.244338 7829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "939b841219464bd7b630901961a5144e"
format_stamp: "Formatted at 2026-09-26 13:04:12 on dist-test-slave-j4lq"
server_key: "4409fabdf2f5c52d051773232c9bfbcc"
server_key_iv: "63db12a8a9f296c00cd1b2bbca8441bb"
server_key_version: "encryptionkey@0"
I20260926 13:04:12.244423 7829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:12.265420 7829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:12.265717 7829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:12.265879 7829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:12.270270 7829 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:46481
I20260926 13:04:12.270295 7895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:46481 every 8 connection(s)
I20260926 13:04:12.270651 7829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:12.271621 7896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:12.273846 7896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e: Bootstrap starting.
I20260926 13:04:12.274330 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7829
I20260926 13:04:12.274410 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:12.274422 7896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:12.274597 2392 external_mini_cluster.cc:1428] Setting key 6e23d097d8dfef072f3d590906b1d1e6
I20260926 13:04:12.274770 7896 log.cc:824] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:12.275518 7896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e: No bootstrap required, opened a new log
I20260926 13:04:12.277220 7896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } }
I20260926 13:04:12.277379 7896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:12.277434 7896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939b841219464bd7b630901961a5144e, State: Initialized, Role: FOLLOWER
I20260926 13:04:12.277559 7896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } }
I20260926 13:04:12.277635 7896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:12.277658 7896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:12.277720 7896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:12.278236 7896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } }
I20260926 13:04:12.278352 7896 leader_election.cc:302] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 939b841219464bd7b630901961a5144e; no voters:
I20260926 13:04:12.278501 7896 leader_election.cc:288] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:12.278546 7901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:12.278663 7901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [term 1 LEADER]: Becoming Leader. State: Replica: 939b841219464bd7b630901961a5144e, State: Running, Role: LEADER
I20260926 13:04:12.278757 7896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:12.278739 7901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } }
I20260926 13:04:12.279049 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 939b841219464bd7b630901961a5144e. Latest consensus state: current_term: 1 leader_uuid: "939b841219464bd7b630901961a5144e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } } }
I20260926 13:04:12.279111 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:12.279296 7905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:12.279783 7905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:12.280946 7902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "939b841219464bd7b630901961a5144e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "939b841219464bd7b630901961a5144e" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46481 } } }
I20260926 13:04:12.281018 7902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:12.282814 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:12.275070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.2.86.62:46481)
0926 13:04:12.275284 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:12.275294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:12.275327 (+ 33us) server_negotiation.cc:407] Connection header received
0926 13:04:12.277191 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:12.277201 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:12.277322 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:12.277461 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:12.277767 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.279318 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.280424 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.281026 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.281171 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.281959 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:12.281963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:12.281963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:12.282178 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:12.282223 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:12.282238 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:12.282308 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:12.282377 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 13:04:12.282549 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260926 13:04:12.283788 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:04:12.284926 7918 mvcc.cc:205] Tried to move back new op lower bound from 7333592482941661184 to 7333592482934116352. Current Snapshot: MvccSnapshot[applied={T|T < 7333592482941661184}]
I20260926 13:04:12.285373 7905 catalog_manager.cc:1347] Generated new cluster ID: 3dc043108adc4500be62d6c19d77fb7d
I20260926 13:04:12.285434 7905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:12.300860 7905 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:12.301620 7905 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:12.321740 7905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 939b841219464bd7b630901961a5144e: Generated new TSK 0
I20260926 13:04:12.321980 7905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:12.378376 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:12.330780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.2.86.62:46481)
0926 13:04:12.330842 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:12.330844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:12.330935 (+ 91us) server_negotiation.cc:407] Connection header received
0926 13:04:12.330990 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:12.330991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:12.331003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:12.331087 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0926 13:04:12.331200 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.332000 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.332734 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.333060 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.333134 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.333274 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 13:04:12.378084 (+ 44810us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 13:04:12.378297 (+ 213us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 13:04:12.378453 7900 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 13:04:12.378486 7925 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:12.330703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:46481 (local address 127.0.0.1:33174)
0926 13:04:12.330870 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:12.330879 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:12.330931 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:12.331090 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:12.331091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:12.331092 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0926 13:04:12.331143 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:12.331144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.332081 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.332081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:12.332655 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:12.332656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:12.333130 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:12.333132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:12.333181 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:12.333186 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 13:04:12.378246 (+ 45060us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 13:04:12.378381 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.86.62:46481: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260926 13:04:12.378892 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7829
2026-09-26T13:04:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (252 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:12 dist-test-slave-j4lq krb5kdc[7937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:12 dist-test-slave-j4lq krb5kdc[7937](info): set up 2 sockets
Sep 26 13:04:12 dist-test-slave-j4lq krb5kdc[7937](info): commencing operation
krb5kdc: starting...
W20260926 13:04:13.561074 2392 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:13 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:13.671595 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:32783
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:32783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:13.748618 7950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:13.748847 7950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:13.748885 7950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:13.750018 7950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:13.750085 7950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:13.750108 7950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:13.750129 7950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:13.750159 7950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:13.751529 7950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:32783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:13.751797 7950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:13.752054 7950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:13.756628 7958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:13.756786 7950 server_base.cc:1034] running on GCE node
W20260926 13:04:13.756605 7955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:13.756597 7956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:13.757128 7950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:13.757408 7950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:13.758574 7950 hybrid_clock.cc:648] HybridClock initialized: now 1790427853758554 us; error 36 us; skew 500 ppm
Sep 26 13:04:13 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427853, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:13.767241 7950 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:13.767700 7950 webserver.cc:460] Webserver started at http://127.2.86.62:42095/ using document root <none> and password file <none>
I20260926 13:04:13.767920 7950 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:13.767985 7950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:13.768116 7950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:13.769181 7950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "28088faacb1a40ba9c32069834e88507"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "02ba17336f464871ead36c5c4227e28a"
server_key_iv: "dc68ab55d84c0ce4c4a69e2729466b8c"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.769531 7950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "28088faacb1a40ba9c32069834e88507"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "02ba17336f464871ead36c5c4227e28a"
server_key_iv: "dc68ab55d84c0ce4c4a69e2729466b8c"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.770807 7950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:13.771484 7965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:13.771642 7950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:13.771752 7950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "28088faacb1a40ba9c32069834e88507"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "02ba17336f464871ead36c5c4227e28a"
server_key_iv: "dc68ab55d84c0ce4c4a69e2729466b8c"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.771831 7950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:13.780368 7950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:13.782459 7950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:13.782608 7950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:13.786413 7950 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:32783
I20260926 13:04:13.786441 8017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:32783 every 8 connection(s)
I20260926 13:04:13.786791 7950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:13.787554 8018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:13.789678 8018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507: Bootstrap starting.
I20260926 13:04:13.790160 8018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:13.790378 8018 log.cc:824] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:13.790951 8018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507: No bootstrap required, opened a new log
I20260926 13:04:13.792366 8018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } }
I20260926 13:04:13.792483 8018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:13.792505 8018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28088faacb1a40ba9c32069834e88507, State: Initialized, Role: FOLLOWER
I20260926 13:04:13.792595 8018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } }
I20260926 13:04:13.792670 8018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:13.792702 8018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:13.792749 8018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:13.793255 8018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } }
I20260926 13:04:13.793367 8018 leader_election.cc:302] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28088faacb1a40ba9c32069834e88507; no voters:
I20260926 13:04:13.793534 8018 leader_election.cc:288] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:13.793565 8021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:13.793766 8018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:13.793876 8021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [term 1 LEADER]: Becoming Leader. State: Replica: 28088faacb1a40ba9c32069834e88507, State: Running, Role: LEADER
I20260926 13:04:13.793978 8021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } }
I20260926 13:04:13.794344 8022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28088faacb1a40ba9c32069834e88507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } } }
I20260926 13:04:13.794461 8022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:13.794341 8023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28088faacb1a40ba9c32069834e88507. Latest consensus state: current_term: 1 leader_uuid: "28088faacb1a40ba9c32069834e88507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28088faacb1a40ba9c32069834e88507" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32783 } } }
I20260926 13:04:13.794624 8023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:13.795120 8030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:13.795608 8030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:13.795681 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 7950
I20260926 13:04:13.795785 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:13.795979 2392 external_mini_cluster.cc:1428] Setting key 28903d19456c625bc0f94676680dc8a0
I20260926 13:04:13.797147 8030 catalog_manager.cc:1347] Generated new cluster ID: f78d3029a5154dd49c9aa26a95ab6fbf
I20260926 13:04:13.797194 8030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:13 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:13.810922 8030 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:13.811167 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:13.796373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.2.86.62:32783)
0926 13:04:13.796509 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:13.796519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:13.796687 (+ 168us) server_negotiation.cc:407] Connection header received
0926 13:04:13.796833 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:13.796839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:13.796918 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:13.801000 (+ 4082us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:13.801116 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.802822 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.803853 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.804477 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.804590 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:13.807358 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:13.807362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:13.807362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:13.807373 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:13.809249 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:13.809634 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:13.809637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:13.809637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:13.809749 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:13.809977 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:13.809978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:13.809978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:13.810257 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:13.810393 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:13.810678 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 13:04:13.810852 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":79,"threads_started":1}
I20260926 13:04:13.811794 8030 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:13.817176 8030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28088faacb1a40ba9c32069834e88507: Generated new TSK 0
I20260926 13:04:13.817327 8030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:13.841867 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:13.924347 8044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:13.924575 8044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:13.924614 8044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:13.925848 8044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:13.925907 8044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:13.925953 8044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:13.927426 8044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:13.927686 8044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:13.927913 8044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:13.932734 8052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:13.932734 8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:13.932746 8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:13.933007 8044 server_base.cc:1034] running on GCE node
I20260926 13:04:13.933228 8044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:13.933480 8044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:13.934737 8044 hybrid_clock.cc:648] HybridClock initialized: now 1790427853934715 us; error 33 us; skew 500 ppm
Sep 26 13:04:13 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427853, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:13.937281 8044 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:13.937690 8044 webserver.cc:460] Webserver started at http://127.2.86.1:46609/ using document root <none> and password file <none>
I20260926 13:04:13.937909 8044 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:13.937985 8044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:13.938131 8044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:13.939138 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "cf8e216de0de40b692952762fb56dc06"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "31e3599d0f7f7174878bdf5e3448e006"
server_key_iv: "33576eaa6ad0fcf219adb1314db94d4d"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.939481 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "cf8e216de0de40b692952762fb56dc06"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "31e3599d0f7f7174878bdf5e3448e006"
server_key_iv: "33576eaa6ad0fcf219adb1314db94d4d"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.940665 8044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:13.941457 8059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:13.941622 8044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:13.941711 8044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "cf8e216de0de40b692952762fb56dc06"
format_stamp: "Formatted at 2026-09-26 13:04:13 on dist-test-slave-j4lq"
server_key: "31e3599d0f7f7174878bdf5e3448e006"
server_key_iv: "33576eaa6ad0fcf219adb1314db94d4d"
server_key_version: "encryptionkey@0"
I20260926 13:04:13.941809 8044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:13.963065 8044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:13.964993 8044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:13.965137 8044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:13.965355 8044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:13.965667 8044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:13.965719 8044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:13.965759 8044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:13.965788 8044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:13.971751 8044 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:42383
I20260926 13:04:13.971787 8172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:42383 every 8 connection(s)
I20260926 13:04:13.972124 8044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:13.976365 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8044
I20260926 13:04:13.976467 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:13.976655 2392 external_mini_cluster.cc:1428] Setting key 1bc973b725555b5eada1f5741e62ca2c
Sep 26 13:04:13 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427853, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:13.981804 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:13.972859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:45617 (local address 127.2.86.62:32783)
0926 13:04:13.972933 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:13.972938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:13.973145 (+ 207us) server_negotiation.cc:407] Connection header received
0926 13:04:13.974143 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:13.974145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:13.974162 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:13.974217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:13.974850 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.975350 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.976348 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.976640 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.976725 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:13.978859 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:13.978861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:13.978861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:13.978868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:13.980220 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:13.980700 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:13.980701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:13.980701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:13.980760 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:13.981045 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:13.981046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:13.981046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:13.981152 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:13.981220 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:13.981487 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 13:04:13.981569 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:04:13.981851 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:13.972850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32783 (local address 127.2.86.1:45617)
0926 13:04:13.973032 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:13.973065 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:13.974041 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:13.974305 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:13.974308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:13.974553 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:13.974731 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:13.974735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.975460 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.975461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:13.976246 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:13.976248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:13.976732 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:13.976733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:13.976927 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:13.977463 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:13.977473 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:13.978755 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:13.980320 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:13.980327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:13.980329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:13.980640 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:13.980824 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:13.980825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:13.980825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:13.980969 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:13.981222 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:13.981229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:13.981358 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 13:04:13.981453 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 13:04:13.981596 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260926 13:04:13.982352 8173 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32783
I20260926 13:04:13.982456 8173 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:13.982617 8173 heartbeater.cc:511] Master 127.2.86.62:32783 requested a full tablet report, sending...
I20260926 13:04:13.983165 7982 ts_manager.cc:194] Registered new tserver with Master: cf8e216de0de40b692952762fb56dc06 (127.2.86.1:42383)
I20260926 13:04:13.984124 7982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:45617
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:13.999467 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:14.076897 8178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:14.077081 8178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:14.077104 8178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:14.078157 8178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:14.078189 8178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:14.078214 8178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:14.079587 8178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:14.079758 8178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:14.079918 8178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:04:14.084893 8178 server_base.cc:1034] running on GCE node
W20260926 13:04:14.084843 8184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:14.084830 8186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:14.084843 8183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:14.085242 8178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:14.085438 8178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:14.086572 8178 hybrid_clock.cc:648] HybridClock initialized: now 1790427854086563 us; error 25 us; skew 500 ppm
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.089004 8178 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:14.089399 8178 webserver.cc:460] Webserver started at http://127.2.86.2:44567/ using document root <none> and password file <none>
I20260926 13:04:14.089619 8178 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:14.089684 8178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:14.089812 8178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:14.090793 8178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "139deba6d8f6469da4828c7552094b8f"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "bda84e3fff9da0d8b940b273c51a49af"
server_key_iv: "eed6b1643a7bf75c02ae50e674ed425f"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.091142 8178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "139deba6d8f6469da4828c7552094b8f"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "bda84e3fff9da0d8b940b273c51a49af"
server_key_iv: "eed6b1643a7bf75c02ae50e674ed425f"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.092350 8178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:14.093078 8193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.093232 8178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.093300 8178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "139deba6d8f6469da4828c7552094b8f"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "bda84e3fff9da0d8b940b273c51a49af"
server_key_iv: "eed6b1643a7bf75c02ae50e674ed425f"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.093348 8178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:14.101840 8178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:14.103541 8178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:14.103644 8178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:14.103801 8178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:14.104099 8178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:14.104171 8178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.104234 8178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:14.104265 8178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.109829 8178 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:45381
I20260926 13:04:14.109931 8306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:45381 every 8 connection(s)
I20260926 13:04:14.110230 8178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:14.113356 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8178
I20260926 13:04:14.113468 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:14.113651 2392 external_mini_cluster.cc:1428] Setting key 97826415d5b78af2936a9859ef306385
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:14.120339 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.110986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51329 (local address 127.2.86.62:32783)
0926 13:04:14.111051 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.111053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.111222 (+ 169us) server_negotiation.cc:407] Connection header received
0926 13:04:14.112115 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.112116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.112125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.112185 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.112731 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.113381 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.114627 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.114951 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.115025 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.117256 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:14.117258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:14.117258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:14.117266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:14.118995 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.119367 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.119368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.119369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.119437 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.119666 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.119667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.119667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.119813 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:14.119888 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:14.120141 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 13:04:14.120234 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:04:14.120350 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.110946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32783 (local address 127.2.86.2:51329)
0926 13:04:14.111100 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:14.111140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:14.112033 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:14.112239 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:14.112242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:14.112487 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:14.112664 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:14.112668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.113494 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.113495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:14.114548 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:14.114550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.115032 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.115033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:14.115125 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.115637 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:14.115645 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:14.117169 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:14.119080 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:14.119085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:14.119087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:14.119292 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:14.119506 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:14.119507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:14.119507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:14.119598 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:14.119881 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:14.119885 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:14.120014 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 13:04:14.120094 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:04:14.120203 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260926 13:04:14.120800 8307 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32783
I20260926 13:04:14.120910 8307 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:14.121084 8307 heartbeater.cc:511] Master 127.2.86.62:32783 requested a full tablet report, sending...
I20260926 13:04:14.121469 7982 ts_manager.cc:194] Registered new tserver with Master: 139deba6d8f6469da4828c7552094b8f (127.2.86.2:45381)
I20260926 13:04:14.121992 7982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:51329
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:14.137660 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:14.215960 8312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:14.216140 8312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:14.216161 8312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:14.217365 8312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:14.217406 8312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:14.217433 8312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:14.218824 8312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:14.219053 8312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:14.219291 8312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:04:14.224231 8312 server_base.cc:1034] running on GCE node
W20260926 13:04:14.224185 8317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:14.224334 8318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:14.224367 8320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:14.224581 8312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:14.224939 8312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:14.226130 8312 hybrid_clock.cc:648] HybridClock initialized: now 1790427854226110 us; error 34 us; skew 500 ppm
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.228600 8312 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:14.229107 8312 webserver.cc:460] Webserver started at http://127.2.86.3:43367/ using document root <none> and password file <none>
I20260926 13:04:14.229328 8312 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:14.229395 8312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:14.229543 8312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:14.230553 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "c38dda841c874acab0f53f21efa7f272"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "7382ebe085984fd38efea9f8c152142f"
server_key_iv: "770bae0f3607833c68fa5fdf9c265573"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.230923 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "c38dda841c874acab0f53f21efa7f272"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "7382ebe085984fd38efea9f8c152142f"
server_key_iv: "770bae0f3607833c68fa5fdf9c265573"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.232170 8312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:14.232842 8327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.233028 8312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:14.233124 8312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "c38dda841c874acab0f53f21efa7f272"
format_stamp: "Formatted at 2026-09-26 13:04:14 on dist-test-slave-j4lq"
server_key: "7382ebe085984fd38efea9f8c152142f"
server_key_iv: "770bae0f3607833c68fa5fdf9c265573"
server_key_version: "encryptionkey@0"
I20260926 13:04:14.233203 8312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:14.242314 8312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:14.243964 8312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:14.244120 8312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:14.244333 8312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:14.244678 8312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:14.244729 8312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.244768 8312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:14.244802 8312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:14.250545 8312 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:34269
I20260926 13:04:14.250592 8440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:34269 every 8 connection(s)
I20260926 13:04:14.250880 8312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:14.251909 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8312
I20260926 13:04:14.252033 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:14.252225 2392 external_mini_cluster.cc:1428] Setting key 59a8c1caafb265f9a4d483d2eb783e05
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:14.259811 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.251663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:59073 (local address 127.2.86.62:32783)
0926 13:04:14.251756 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.251758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.251952 (+ 194us) server_negotiation.cc:407] Connection header received
0926 13:04:14.253318 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.253318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.253327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.253371 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.253923 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.254406 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.255340 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.255540 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.255602 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.257582 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:14.257583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:14.257583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:14.257589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:14.258747 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.259099 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.259100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.259100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.259145 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.259341 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.259341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.259341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.259438 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:14.259485 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:14.259698 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 13:04:14.259753 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 13:04:14.259966 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.251622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32783 (local address 127.2.86.3:59073)
0926 13:04:14.251810 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:14.251859 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:14.253211 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:14.253422 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:14.253425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:14.253671 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:14.253843 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:14.253847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.254484 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.254484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:14.255244 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:14.255245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.255619 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.255620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:14.255713 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.256121 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:14.256128 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:14.257500 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:14.258825 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:14.258830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:14.258832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:14.259036 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:14.259200 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:14.259201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:14.259201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:14.259289 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:14.259489 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:14.259493 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:14.259593 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 13:04:14.259677 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 13:04:14.259782 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":150,"threads_started":1}
I20260926 13:04:14.260504 8441 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32783
I20260926 13:04:14.260574 8441 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:14.260692 8441 heartbeater.cc:511] Master 127.2.86.62:32783 requested a full tablet report, sending...
I20260926 13:04:14.261042 7982 ts_manager.cc:194] Registered new tserver with Master: c38dda841c874acab0f53f21efa7f272 (127.2.86.3:34269)
I20260926 13:04:14.261394 7982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:59073
I20260926 13:04:14.263901 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/test-admin.keytab.
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.278105 2392 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 13:04:14.279577 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:14.278849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.2.86.1:42383)
0926 13:04:14.279006 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.279011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.279022 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:04:14.279050 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.279051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.279064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.279143 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.279497 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48384: BlockingRecv error: recv got EOF from 127.0.0.1:48384 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
W20260926 13:04:14.279680 2392 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc' of type FILE
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.280552 2392 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.281530 2392 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:04:14.288395 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.281977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.2.86.1:42383)
0926 13:04:14.282035 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.282037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.282090 (+ 53us) server_negotiation.cc:407] Connection header received
0926 13:04:14.282151 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.282152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.282162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.282210 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.282656 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.283315 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.283985 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.284290 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.284345 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.286194 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:14.286200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:14.286200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:14.286206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:14.287541 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.287823 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.287825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.287825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.287883 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.288019 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.288019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.288019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.288117 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:14.288199 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:14.288275 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 13:04:14.288327 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 13:04:14.289839 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:14.289446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.2.86.1:42383)
0926 13:04:14.289476 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.289480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.289493 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:04:14.289541 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.289541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.289549 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.289592 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.289799 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48398: BlockingRecv error: recv got EOF from 127.0.0.1:48398 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 13:04:14.289971 2392 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc)
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.290898 2392 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.291673 2392 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:04:14.298326 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.292069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.2.86.1:42383)
0926 13:04:14.292139 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.292140 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.292173 (+ 33us) server_negotiation.cc:407] Connection header received
0926 13:04:14.292217 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.292218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.292224 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.292265 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.292561 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.293149 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.293862 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.294116 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.294166 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.296176 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:14.296177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:14.296177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:14.296183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:14.297420 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.297735 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.297736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.297736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.297778 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.297928 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.297929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.297929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.298014 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:14.298077 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:14.298153 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 13:04:14.298201 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 13:04:14.299893 8448 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:14.299350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.2.86.1:42383)
0926 13:04:14.299417 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.299419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.299428 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:14.299492 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.299493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.299499 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.299541 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.299856 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48416: BlockingRecv error: recv got EOF from 127.0.0.1:48416 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260926 13:04:14.299964 2392 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790427826505950-2392-0/krb5kdc/krb5cc' of type FILE
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.300834 2392 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:14.301666 2392 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[7937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.1@KRBTEST.COM
I20260926 13:04:14.307509 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:14.302089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.2.86.1:42383)
0926 13:04:14.302140 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:14.302142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:14.302179 (+ 37us) server_negotiation.cc:407] Connection header received
0926 13:04:14.302223 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:14.302223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:14.302230 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:14.302275 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:14.302597 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.303031 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.303629 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:14.303792 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:14.303837 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:14.305618 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:14.305619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:14.305619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:14.305625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:14.306711 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.306996 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.306997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.306997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.307039 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:14.307191 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:14.307192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:14.307192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:14.307266 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:14.307328 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:14.307408 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 13:04:14.307461 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:14.308090 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8044
I20260926 13:04:14.312583 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8178
I20260926 13:04:14.316505 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8312
I20260926 13:04:14.320772 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 7950
2026-09-26T13:04:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1773 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[8461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[8461](info): set up 2 sockets
Sep 26 13:04:14 dist-test-slave-j4lq krb5kdc[8461](info): commencing operation
krb5kdc: starting...
W20260926 13:04:15.345047 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab.
Entry for principal oryx/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab.
I20260926 13:04:15.457906 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33835
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab
--principal=oryx/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:15.536835 8474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:15.537087 8474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:15.537123 8474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:15.538240 8474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:15.538305 8474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:15.538331 8474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:15.538362 8474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:15.538384 8474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:15.539901 8474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab
--principal=oryx/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:15.540162 8474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:15.540377 8474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:15.545082 8480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.545066 8479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.545629 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:15.545765 8474 server_base.cc:1034] running on GCE node
I20260926 13:04:15.546033 8474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:15.546262 8474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:15.547407 8474 hybrid_clock.cc:648] HybridClock initialized: now 1790427855547395 us; error 28 us; skew 500 ppm
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:15.549937 8474 init.cc:377] Logged in from keytab as oryx/127.2.86.62@KRBTEST.COM (short username oryx)
I20260926 13:04:15.550346 8474 webserver.cc:460] Webserver started at http://127.2.86.62:34365/ using document root <none> and password file <none>
I20260926 13:04:15.550591 8474 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:15.550658 8474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:15.550810 8474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:15.551892 8474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "35ada61a0f264683b84d554ec77b1949"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "079ac40e86596465bbd520ff690a95e2"
server_key_iv: "29f35f46cae50cd8975cb771c4dda7d7"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.552239 8474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "35ada61a0f264683b84d554ec77b1949"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "079ac40e86596465bbd520ff690a95e2"
server_key_iv: "29f35f46cae50cd8975cb771c4dda7d7"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.553610 8474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:15.554317 8489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.554500 8474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.554593 8474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "35ada61a0f264683b84d554ec77b1949"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "079ac40e86596465bbd520ff690a95e2"
server_key_iv: "29f35f46cae50cd8975cb771c4dda7d7"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.554668 8474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:15.582043 8474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:15.583822 8474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:15.583969 8474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:15.587795 8474 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33835
I20260926 13:04:15.587833 8541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33835 every 8 connection(s)
I20260926 13:04:15.588191 8474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:15.589149 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:15.591234 8542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Bootstrap starting.
I20260926 13:04:15.591780 8542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:15.592000 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8474
I20260926 13:04:15.592101 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:15.592103 8542 log.cc:824] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:15.592304 2392 external_mini_cluster.cc:1428] Setting key 2db0ee24ac734e4f91ff0ad54320bfc8
I20260926 13:04:15.592886 8542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: No bootstrap required, opened a new log
I20260926 13:04:15.595419 8542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } }
I20260926 13:04:15.595579 8542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:15.595611 8542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ada61a0f264683b84d554ec77b1949, State: Initialized, Role: FOLLOWER
I20260926 13:04:15.595726 8542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } }
I20260926 13:04:15.595816 8542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:15.595844 8542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:15.595885 8542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:15.596444 8542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } }
I20260926 13:04:15.596560 8542 leader_election.cc:302] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ada61a0f264683b84d554ec77b1949; no voters:
I20260926 13:04:15.596815 8542 leader_election.cc:288] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:15.596817 8547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:15.597007 8547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 LEADER]: Becoming Leader. State: Replica: 35ada61a0f264683b84d554ec77b1949, State: Running, Role: LEADER
I20260926 13:04:15.597084 8547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } }
I20260926 13:04:15.597122 8542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:15.597467 8547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ada61a0f264683b84d554ec77b1949. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } }
I20260926 13:04:15.597610 8547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:15.598064 8554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:15.598557 8554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:15.599255 8548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } }
I20260926 13:04:15.599390 8548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:15.600705 8554 catalog_manager.cc:1347] Generated new cluster ID: 427445596ec04391957d4f3b2741f395
I20260926 13:04:15.600755 8554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:15.603209 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:15.592791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.2.86.62:33835)
0926 13:04:15.592965 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:15.592974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:15.593012 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:04:15.593401 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:15.593406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:15.593446 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:15.593552 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:15.594305 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.595119 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.596148 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.596519 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.596642 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:15.599468 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:15.599472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:15.599472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:15.599484 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:15.601516 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.601876 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.601879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.601879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.601968 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.602128 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.602129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.602129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.602382 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:15.602511 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:15.602910 (+ 399us) server_negotiation.cc:299] Negotiation successful
0926 13:04:15.603025 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":95,"threads_started":1}
I20260926 13:04:15.615630 8554 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:15.616127 8554 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:15.622318 8554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Generated new TSK 0
I20260926 13:04:15.622483 8554 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
I20260926 13:04:15.653734 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:15.735646 8568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:15.735872 8568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:15.735910 8568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:15.737108 8568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:15.737174 8568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:15.737216 8568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:15.738636 8568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:15.738875 8568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:15.739087 8568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:15.743820 8576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.743829 8573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.743820 8574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:15.743953 8568 server_base.cc:1034] running on GCE node
I20260926 13:04:15.744266 8568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:15.744459 8568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:15.745585 8568 hybrid_clock.cc:648] HybridClock initialized: now 1790427855745570 us; error 34 us; skew 500 ppm
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:15.747927 8568 init.cc:377] Logged in from keytab as oryx/127.2.86.1@KRBTEST.COM (short username oryx)
I20260926 13:04:15.748373 8568 webserver.cc:460] Webserver started at http://127.2.86.1:36359/ using document root <none> and password file <none>
I20260926 13:04:15.748577 8568 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:15.748618 8568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:15.748749 8568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:15.749747 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "6bb0b28f0ff130048efc37b92ed88c6d"
server_key_iv: "3b28013416e494c581d8763411e5f8c1"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.750087 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "6bb0b28f0ff130048efc37b92ed88c6d"
server_key_iv: "3b28013416e494c581d8763411e5f8c1"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.751283 8568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:15.751940 8583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.752101 8568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:15.752192 8568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "6bb0b28f0ff130048efc37b92ed88c6d"
server_key_iv: "3b28013416e494c581d8763411e5f8c1"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.752266 8568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:15.763723 8568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:15.765509 8568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:15.765661 8568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:15.765868 8568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:15.766168 8568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:15.766222 8568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.766263 8568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:15.766290 8568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.772089 8568 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41749
I20260926 13:04:15.772122 8696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41749 every 8 connection(s)
I20260926 13:04:15.772459 8568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:15.778683 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8568
I20260926 13:04:15.778791 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:15.779006 2392 external_mini_cluster.cc:1428] Setting key 419a98a525db1a2ea4d61d9304f2a647
I20260926 13:04:15.781783 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:15.773191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:58797 (local address 127.2.86.62:33835)
0926 13:04:15.773266 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:15.773269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:15.773463 (+ 194us) server_negotiation.cc:407] Connection header received
0926 13:04:15.773912 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:15.773913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:15.773923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:15.773974 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:15.774823 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.775344 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.776259 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.776459 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.776508 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:15.778656 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:15.778658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:15.778658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:15.778667 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:15.780415 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.780806 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.780807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.780808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.780898 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.781124 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.781125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.781126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.781274 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:15.781348 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:15.781656 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 13:04:15.781720 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:04:15.782065 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:15.773183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:58797)
0926 13:04:15.773348 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:15.773381 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:15.773829 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:15.774025 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:15.774029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:15.774554 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:15.774742 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:15.774746 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.775420 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.775420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:15.776190 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:15.776192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.776526 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.776527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:15.776619 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:15.777209 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:15.777217 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:15.778560 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:15.780498 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:15.780503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:15.780505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:15.780723 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:15.780964 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:15.780965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:15.780965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:15.781053 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:15.781347 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:15.781352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:15.781518 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 13:04:15.781637 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:04:15.781807 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260926 13:04:15.782627 8697 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:15.782719 8697 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:15.782910 8697 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:15.783457 8506 ts_manager.cc:194] Registered new tserver with Master: c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:15.784101 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:58797
WARNING: no policy specified for oryx/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
I20260926 13:04:15.804177 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:15.884438 8702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:15.884652 8702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:15.884691 8702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:15.885946 8702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:15.886014 8702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:15.886058 8702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:15.887440 8702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:15.887689 8702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:15.887897 8702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:15.892709 8708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.892735 8707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:15.892762 8710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:15.892962 8702 server_base.cc:1034] running on GCE node
I20260926 13:04:15.893159 8702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:15.893404 8702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:15.894554 8702 hybrid_clock.cc:648] HybridClock initialized: now 1790427855894541 us; error 27 us; skew 500 ppm
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:15.896992 8702 init.cc:377] Logged in from keytab as oryx/127.2.86.2@KRBTEST.COM (short username oryx)
I20260926 13:04:15.897493 8702 webserver.cc:460] Webserver started at http://127.2.86.2:35967/ using document root <none> and password file <none>
I20260926 13:04:15.897722 8702 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:15.897785 8702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:15.897943 8702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:15.898917 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "b304cd6207334e4e8d01d0f95a14cea3"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "0708a3b7db0005d6b57e21b95304213c"
server_key_iv: "42abb62d7da26605cef497fe3abe99ed"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.899318 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "b304cd6207334e4e8d01d0f95a14cea3"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "0708a3b7db0005d6b57e21b95304213c"
server_key_iv: "42abb62d7da26605cef497fe3abe99ed"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.900555 8702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:15.901305 8717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.901477 8702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.901571 8702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "b304cd6207334e4e8d01d0f95a14cea3"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "0708a3b7db0005d6b57e21b95304213c"
server_key_iv: "42abb62d7da26605cef497fe3abe99ed"
server_key_version: "encryptionkey@0"
I20260926 13:04:15.901654 8702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:15.913017 8702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:15.915237 8702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:15.915391 8702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:15.915599 8702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:15.915894 8702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:15.915944 8702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.915992 8702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:15.916021 8702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:15.921653 8702 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:41749
I20260926 13:04:15.921697 8830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:41749 every 8 connection(s)
I20260926 13:04:15.921984 8702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:04:15 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:15.928639 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8702
I20260926 13:04:15.928761 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:15.928996 2392 external_mini_cluster.cc:1428] Setting key 2d22899df12a2ffc9f540b93792e0b16
I20260926 13:04:15.931049 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:15.922641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35725 (local address 127.2.86.62:33835)
0926 13:04:15.922683 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:15.922685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:15.922868 (+ 183us) server_negotiation.cc:407] Connection header received
0926 13:04:15.923327 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:15.923327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:15.923335 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:15.923388 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:15.924229 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.924747 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.925723 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.925915 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.925966 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:15.928028 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:15.928030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:15.928030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:15.928037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:15.929603 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.930097 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.930098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.930098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.930149 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:15.930432 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:15.930432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:15.930433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:15.930531 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:15.930580 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:15.930920 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 13:04:15.930987 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:04:15.931344 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:15.922599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:35725)
0926 13:04:15.922743 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:15.922782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:15.923251 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:15.923445 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:15.923448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:15.923975 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:15.924150 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:15.924154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.924825 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.924825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:15.925649 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:15.925651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:15.925993 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:15.925994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:15.926088 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:15.926579 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:15.926587 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:15.927936 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:15.929693 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:15.929700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:15.929702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:15.930026 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:15.930213 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:15.930214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:15.930215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:15.930357 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:15.930594 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:15.930599 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:15.930790 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 13:04:15.930902 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 13:04:15.931075 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260926 13:04:15.931806 8831 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:15.931921 8831 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:15.932103 8831 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:15.932519 8506 ts_manager.cc:194] Registered new tserver with Master: b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749)
I20260926 13:04:15.933079 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:35725
WARNING: no policy specified for oryx/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
I20260926 13:04:15.955284 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:16.033282 8836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:16.033515 8836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:16.033552 8836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:16.034684 8836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:16.034749 8836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:16.034788 8836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:16.036238 8836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:16.036496 8836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:16.036722 8836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.041663 8844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.041699 8841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.041663 8842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.041880 8836 server_base.cc:1034] running on GCE node
I20260926 13:04:16.042058 8836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.042294 8836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.043448 8836 hybrid_clock.cc:648] HybridClock initialized: now 1790427856043430 us; error 33 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.045893 8836 init.cc:377] Logged in from keytab as oryx/127.2.86.3@KRBTEST.COM (short username oryx)
I20260926 13:04:16.046309 8836 webserver.cc:460] Webserver started at http://127.2.86.3:38067/ using document root <none> and password file <none>
I20260926 13:04:16.046559 8836 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.046627 8836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.046772 8836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:16.047770 8836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "763bb84a9d2b4a828a54b3a5f7b3358b"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "b2e0f9169bfa33a9f7597189d1ed74c0"
server_key_iv: "50ec967310ce9da37958d2e98ccb09c5"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.048130 8836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "763bb84a9d2b4a828a54b3a5f7b3358b"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "b2e0f9169bfa33a9f7597189d1ed74c0"
server_key_iv: "50ec967310ce9da37958d2e98ccb09c5"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.049600 8836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:16.050313 8851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.050480 8836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.050580 8836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "763bb84a9d2b4a828a54b3a5f7b3358b"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "b2e0f9169bfa33a9f7597189d1ed74c0"
server_key_iv: "50ec967310ce9da37958d2e98ccb09c5"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.050649 8836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.064988 8836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.066847 8836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:16.067003 8836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.067220 8836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:16.067523 8836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:16.067575 8836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.067616 8836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:16.067644 8836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.073460 8836 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:45625
I20260926 13:04:16.073504 8964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:45625 every 8 connection(s)
I20260926 13:04:16.073800 8836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:16.079954 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8836
I20260926 13:04:16.080053 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:16.080248 2392 external_mini_cluster.cc:1428] Setting key 98cad33cb1d01983dd735ba3fbc75eea
I20260926 13:04:16.082273 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.074421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38121 (local address 127.2.86.62:33835)
0926 13:04:16.074500 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.074502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.074663 (+ 161us) server_negotiation.cc:407] Connection header received
0926 13:04:16.075114 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.075115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.075125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.075175 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.076023 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.076507 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.077472 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.077646 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.077694 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.079568 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.079570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.079570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.079576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.081261 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.081595 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.081596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.081596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.081641 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.081804 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.081805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.081805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.081898 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.081944 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.082157 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.082212 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:04:16.082433 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.074385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:38121)
0926 13:04:16.074552 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.074585 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.075038 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.075225 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.075229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.075762 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.075944 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.075948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.076581 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.076581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.077400 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.077402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.077712 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.077713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.077809 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.078210 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.078217 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.079494 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.081319 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.081323 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.081325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.081533 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.081679 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.081680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.081680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.081757 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.081951 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.081955 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.082056 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.082133 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.082237 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260926 13:04:16.082871 8965 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:16.082971 8965 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:16.083137 8965 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:16.083493 8506 ts_manager.cc:194] Registered new tserver with Master: 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3:45625)
I20260926 13:04:16.083858 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:38121
I20260926 13:04:16.092121 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.86.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.61.keytab.
Entry for principal oryx/127.2.86.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.61.keytab.
I20260926 13:04:16.115679 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:35189
--webserver_interface=127.2.86.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.61.keytab
--principal=oryx/127.2.86.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33835,127.2.86.61:35189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:16.193457 8970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:16.193645 8970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:16.193666 8970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:16.194679 8970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:16.194710 8970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:16.194723 8970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:16.194736 8970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:16.194747 8970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:16.196046 8970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33835,127.2.86.61:35189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.61.keytab
--principal=oryx/127.2.86.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:35189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.86.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
W20260926 13:04:16.196131 8970 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.86.62:33835,127.2.86.61:35189'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 13:04:16.196226 8970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:16.196375 8970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.201043 8976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.201045 8978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.201105 8975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.201443 8970 server_base.cc:1034] running on GCE node
I20260926 13:04:16.201635 8970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.201851 8970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.203022 8970 hybrid_clock.cc:648] HybridClock initialized: now 1790427856203006 us; error 28 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.205495 8970 init.cc:377] Logged in from keytab as oryx/127.2.86.61@KRBTEST.COM (short username oryx)
I20260926 13:04:16.205878 8970 webserver.cc:460] Webserver started at http://127.2.86.61:40215/ using document root <none> and password file <none>
I20260926 13:04:16.206130 8970 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.206197 8970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.206348 8970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:16.207324 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/instance:
uuid: "25a413f8cd7d48e5890471551cbdfab6"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "378b561dbf77e2fd7d48a260e3b576fb"
server_key_iv: "410070ae606620ada5491cf3b63fd884"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.207665 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal/instance:
uuid: "25a413f8cd7d48e5890471551cbdfab6"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "378b561dbf77e2fd7d48a260e3b576fb"
server_key_iv: "410070ae606620ada5491cf3b63fd884"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.209051 8970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:16.209842 8985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.210009 8970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.210101 8970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "25a413f8cd7d48e5890471551cbdfab6"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "378b561dbf77e2fd7d48a260e3b576fb"
server_key_iv: "410070ae606620ada5491cf3b63fd884"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.210186 8970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.225752 8970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.227941 8970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:16.228108 8970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.231716 8970 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.61:35189
I20260926 13:04:16.231748 9037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.61:35189 every 8 connection(s)
I20260926 13:04:16.232070 8970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
I20260926 13:04:16.233019 9038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:16.234377 9038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.61@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:16.239758 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 8970
I20260926 13:04:16.242944 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.235093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.2.86.62:33835)
0926 13:04:16.235140 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.235142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.235328 (+ 186us) server_negotiation.cc:407] Connection header received
0926 13:04:16.235738 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.235739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.235747 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.235811 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.236309 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.236799 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.237774 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.237956 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.238050 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.239962 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.239964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.239965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.239973 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.241524 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.241963 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.241964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.241965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.242021 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.242280 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.242281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.242281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.242435 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.242513 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.242776 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.242851 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:16.243147 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.235046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45144)
0926 13:04:16.235207 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.235251 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.235664 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.235860 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.235863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.236077 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.236246 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.236249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.236934 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.236935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.237702 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.237704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.238053 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.238054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.238144 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.238583 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.238591 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.239852 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.241598 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.241604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.241606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.241873 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.242084 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.242085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.242085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.242207 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.242502 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.242507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.242639 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.242755 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.242929 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260926 13:04:16.243767 9038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.61@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
I20260926 13:04:16.248720 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.240232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.2.86.61:35189)
0926 13:04:16.240408 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.240414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.240454 (+ 40us) server_negotiation.cc:407] Connection header received
0926 13:04:16.240543 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.240545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.240558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.240640 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.241272 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.241966 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.242712 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.243020 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.243089 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.245760 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.245767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.245767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.245774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.247538 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.247903 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.247906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.247906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.247982 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.248201 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.248202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.248203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.248344 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.248511 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.248551 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.248619 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260926 13:04:16.251219 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.243986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38104)
0926 13:04:16.244031 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.244040 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.244083 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.244273 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.244274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.244427 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.244462 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.244463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.245450 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.245451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.246064 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.246066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.246479 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.246480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.246552 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.246853 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.246858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.248162 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.250320 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.250321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.250321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.250472 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.250666 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.250667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.250667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.250715 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.250940 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.250941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.250973 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.251039 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.251120 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 13:04:16.251302 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.244034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.2.86.61:35189)
0926 13:04:16.244167 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.244169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.244175 (+ 6us) server_negotiation.cc:407] Connection header received
0926 13:04:16.244189 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.244190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.244199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.244273 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.244545 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.245381 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.246139 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.246411 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.246460 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.248248 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.248249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.248249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.248255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.250226 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.250546 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.250547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.250548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.250598 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.250787 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.250788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.250788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.250878 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.250948 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.251060 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.251166 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260926 13:04:16.252288 9038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Bootstrap starting.
I20260926 13:04:16.252858 9038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:16.253278 9038 log.cc:824] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:16.254051 9038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: No bootstrap required, opened a new log
I20260926 13:04:16.255402 9038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } }
I20260926 13:04:16.255534 9038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:16.255568 9038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a413f8cd7d48e5890471551cbdfab6, State: Initialized, Role: FOLLOWER
I20260926 13:04:16.255667 9038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } }
I20260926 13:04:16.255868 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } } }
I20260926 13:04:16.255882 9038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:16.255987 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:04:16.257524 9056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:04:16.257576 9056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:04:16.259085 8970 master_runner.cc:428] Detected that this master 25a413f8cd7d48e5890471551cbdfab6 is joining an existing cluster
I20260926 13:04:16.259147 8970 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.86.61:35189
I20260926 13:04:16.265926 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.260095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.2.86.62:33835)
0926 13:04:16.260138 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.260140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.260147 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:16.260178 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.260179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.260189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.260237 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.260575 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.261098 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.261802 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.261988 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.262055 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.262974 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.262976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.262976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.262982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.264402 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.264900 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.264901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.264902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.264962 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.265336 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.265337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.265338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.265476 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.265554 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.265756 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.265831 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:16.266001 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.259907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45150)
0926 13:04:16.260048 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.260058 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.260103 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.260239 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.260240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.260438 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.260497 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.260498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.261170 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.261171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.261706 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.261708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.262065 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.262066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.262125 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.262511 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.262516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.262897 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.264503 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.264504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.264504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.264784 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.265192 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.265193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.265193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.265261 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.265562 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.265563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.265618 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.265717 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.265852 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260926 13:04:16.267473 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.260066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.2.86.61:35189)
0926 13:04:16.260138 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.260140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.260241 (+ 101us) server_negotiation.cc:407] Connection header received
0926 13:04:16.260316 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.260317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.260325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.260370 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.260698 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.261406 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.262353 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.262682 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.262751 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.263691 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.263693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.263693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.263699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.265847 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.266659 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.266660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.266660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.266710 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.266924 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.266924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.266925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.267038 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.267085 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.267246 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.267376 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":54}
I20260926 13:04:16.267694 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.259951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38120)
0926 13:04:16.260188 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.260196 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.260262 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.260384 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.260385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.260566 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.260619 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.260621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.261490 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.261491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.262277 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.262279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.262771 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.262772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.262834 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.263258 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.263263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.263622 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.265917 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.265918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.265918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.266579 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.266782 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.266783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.266784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.266851 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.267116 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.267116 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.267161 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.267225 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.267600 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":60,"threads_started":1}
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.62@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
I20260926 13:04:16.275439 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.268403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38128)
0926 13:04:16.268497 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.268535 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.268617 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.268742 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.268744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.269037 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.269080 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.269082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.269994 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.269994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.270556 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.270558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.271001 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.271002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.271049 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.271471 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.271474 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.272924 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.274492 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.274497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.274499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.274680 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.274863 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.274864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.274864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.274912 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.275161 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.275166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.275204 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.275285 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.275366 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":53,"threads_started":1}
I20260926 13:04:16.275477 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.268449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.2.86.61:35189)
0926 13:04:16.268486 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.268488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.268608 (+ 120us) server_negotiation.cc:407] Connection header received
0926 13:04:16.268667 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.268667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.268676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.268739 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.269150 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.269817 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.270643 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.270930 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.271002 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.273021 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.273023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.273023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.273029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.274410 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.274746 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.274747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.274747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.274799 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.274977 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.274977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.274978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.275092 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.275153 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.275325 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.275396 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:04:16.276230 8506 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.86.61:35189: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6"
member_type: NON_VOTER
last_known_addr {
host: "127.2.86.61"
port: 35189
}
attrs {
promote: true
}
}
dest_uuid: "35ada61a0f264683b84d554ec77b1949"
cas_config_opid_index: -1
I20260926 13:04:16.276520 8506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } }
I20260926 13:04:16.277014 9065 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } }
I20260926 13:04:16.277284 9012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:16.278067 9065 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.86.61:35189
I20260926 13:04:16.278128 9012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 13:04:16.278254 8970 master.cc:537] Master@127.2.86.61:35189 shutting down...
I20260926 13:04:16.278213 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } }
I20260926 13:04:16.278288 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:16.279697 9066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:04:16.280355 8490 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 13:04:16.280850 8490 consensus_peers.cc:487] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 -> Peer 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61:35189): Couldn't send request to peer 25a413f8cd7d48e5890471551cbdfab6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 13:04:16.281198 8970 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:16.281276 8970 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:16.281330 8970 tablet_replica.cc:330] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: stopping tablet replica
I20260926 13:04:16.293658 8970 master.cc:552] Master@127.2.86.61:35189 shutdown complete.
I20260926 13:04:16.293712 8970 master_runner.cc:305] Clearing existing system tablet
I20260926 13:04:16.293869 8970 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.293907 8970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.294466 8970 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.294970 9070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.295082 8970 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.295136 8970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.295159 8970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "25a413f8cd7d48e5890471551cbdfab6"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "378b561dbf77e2fd7d48a260e3b576fb"
server_key_iv: "410070ae606620ada5491cf3b63fd884"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.295722 8970 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:16.297060 8970 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 13:04:16.297116 8970 log.cc:1196] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 13:04:16.297313 8970 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Deleting consensus metadata
I20260926 13:04:16.297437 8970 master_runner.cc:315] Copying system tablet from 127.2.86.62:33835
I20260926 13:04:16.297995 8970 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: tablet copy: Beginning tablet copy session from remote peer at address 127.2.86.62:33835
I20260926 13:04:16.304346 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.298344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.2.86.62:33835)
0926 13:04:16.298388 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.298390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.298471 (+ 81us) server_negotiation.cc:407] Connection header received
0926 13:04:16.298542 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.298543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.298551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.298592 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.298924 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.299666 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.300714 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.300978 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.301057 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.301997 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.301999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.301999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.302004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.303348 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.303718 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.303718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.303719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.303768 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.303944 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.303944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.303944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.304029 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.304072 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.304239 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.304292 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:04:16.304608 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.298272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45166)
0926 13:04:16.298404 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.298417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.298486 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.298605 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.298605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.298780 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.298842 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.298844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.299749 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.299750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.300617 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.300619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.301068 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.301069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.301141 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.301543 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.301548 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.301906 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.303443 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.303444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.303445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.303652 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.303820 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.303821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.303821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.303877 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.304083 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.304084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.304126 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.304194 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.304284 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":75,"threads_started":1}
I20260926 13:04:16.305125 8526 tablet_copy_service.cc:140] P 35ada61a0f264683b84d554ec77b1949: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 25a413f8cd7d48e5890471551cbdfab6 ({username='oryx', principal='oryx/127.2.86.61@KRBTEST.COM'} at 127.0.0.1:45166)
I20260926 13:04:16.305222 8526 tablet_copy_service.cc:161] P 35ada61a0f264683b84d554ec77b1949: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 25a413f8cd7d48e5890471551cbdfab6 at {username='oryx', principal='oryx/127.2.86.61@KRBTEST.COM'} at 127.0.0.1:45166: session id = 25a413f8cd7d48e5890471551cbdfab6-00000000000000000000000000000000
I20260926 13:04:16.306041 8526 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 13:04:16.306562 8970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 13:04:16.307370 8970 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: tablet copy: Starting download of 0 data blocks...
I20260926 13:04:16.307533 8970 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: tablet copy: Starting download of 1 WAL segments...
I20260926 13:04:16.308389 8970 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 13:04:16.309149 8526 tablet_copy_service.cc:342] P 35ada61a0f264683b84d554ec77b1949: Request end of tablet copy session 25a413f8cd7d48e5890471551cbdfab6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.86.61@KRBTEST.COM'} at 127.0.0.1:45166
I20260926 13:04:16.309203 8526 tablet_copy_service.cc:433] P 35ada61a0f264683b84d554ec77b1949: ending tablet copy session 25a413f8cd7d48e5890471551cbdfab6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 25a413f8cd7d48e5890471551cbdfab6
I20260926 13:04:16.311368 8970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.312485 9077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.312531 9078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.312660 9080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.313028 8970 server_base.cc:1034] running on GCE node
I20260926 13:04:16.313128 8970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.313262 8970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.314383 8970 hybrid_clock.cc:648] HybridClock initialized: now 1790427856314372 us; error 26 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.315706 8970 init.cc:377] Logged in from keytab as oryx/127.2.86.61@KRBTEST.COM (short username oryx)
I20260926 13:04:16.315999 8970 webserver.cc:460] Webserver started at http://127.2.86.61:35815/ using document root <none> and password file <none>
I20260926 13:04:16.316113 8970 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.316169 8970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.316763 8970 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:16.317325 9087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.317485 8970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.317541 8970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "25a413f8cd7d48e5890471551cbdfab6"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "378b561dbf77e2fd7d48a260e3b576fb"
server_key_iv: "410070ae606620ada5491cf3b63fd884"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.317595 8970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.331821 8970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.333415 8970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.336326 8970 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.61:35189
I20260926 13:04:16.336611 9139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.61:35189 every 8 connection(s)
I20260926 13:04:16.336714 8970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
I20260926 13:04:16.337416 9140 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:04:16.337805 9140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Bootstrap starting.
I20260926 13:04:16.340631 9140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:04:16.340858 9140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: Bootstrap complete.
I20260926 13:04:16.341065 9140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } }
I20260926 13:04:16.341147 9140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 25a413f8cd7d48e5890471551cbdfab6, State: Initialized, Role: LEARNER
I20260926 13:04:16.341193 9140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } }
I20260926 13:04:16.341357 9140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:16.341339 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } }
I20260926 13:04:16.341411 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:04:16.342644 9154 catalog_manager.cc:1259] Loaded cluster ID: 427445596ec04391957d4f3b2741f395
I20260926 13:04:16.342695 9154 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: loading cluster ID for follower catalog manager: success
I20260926 13:04:16.343832 9154 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: acquiring CA information for follower catalog manager: success
I20260926 13:04:16.344122 9154 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:04:16.357501 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.350654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.2.86.61:35189)
0926 13:04:16.350758 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.350760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.350767 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:16.350793 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.350794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.350801 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.350870 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.351408 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.352355 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.353104 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.353450 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.353517 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.354734 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.354736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.354736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.354743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.356296 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.356595 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.356597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.356597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.356659 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.356974 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.356975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.356975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.357107 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.357212 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.357304 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.357381 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
WARNING: no policy specified for oryx/127.2.86.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.60.keytab.
Entry for principal oryx/127.2.86.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.60.keytab.
I20260926 13:04:16.383278 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:40093
--webserver_interface=127.2.86.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.60.keytab
--principal=oryx/127.2.86.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:16.468137 9158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:16.468322 9158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:16.468344 9158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:16.469470 9158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:16.469509 9158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:16.469523 9158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:16.469558 9158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:16.469587 9158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:16.471002 9158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.60.keytab
--principal=oryx/127.2.86.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:40093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.86.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:16.471244 9158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:16.471463 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.476079 9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.476083 9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.476141 9158 server_base.cc:1034] running on GCE node
W20260926 13:04:16.476087 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.476459 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.476742 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.477938 9158 hybrid_clock.cc:648] HybridClock initialized: now 1790427856477941 us; error 59 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.480118 9158 init.cc:377] Logged in from keytab as oryx/127.2.86.60@KRBTEST.COM (short username oryx)
I20260926 13:04:16.480520 9158 webserver.cc:460] Webserver started at http://127.2.86.60:43531/ using document root <none> and password file <none>
I20260926 13:04:16.480731 9158 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.480794 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.480949 9158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:16.481910 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/instance:
uuid: "851d33f1a0f3460ab83c1efef939d10c"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "fb4bbbe25516c64db38daa9636a06c0b"
server_key_iv: "350aa0c3e276395de2d0b216e4a8825d"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.482247 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal/instance:
uuid: "851d33f1a0f3460ab83c1efef939d10c"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "fb4bbbe25516c64db38daa9636a06c0b"
server_key_iv: "350aa0c3e276395de2d0b216e4a8825d"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.483457 9158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:04:16.484089 9173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.484242 9158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:16.484332 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "851d33f1a0f3460ab83c1efef939d10c"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "fb4bbbe25516c64db38daa9636a06c0b"
server_key_iv: "350aa0c3e276395de2d0b216e4a8825d"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.484417 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.524528 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.526408 9158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:16.526561 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.530928 9158 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.60:40093
I20260926 13:04:16.530998 9225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.60:40093 every 8 connection(s)
I20260926 13:04:16.531330 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
I20260926 13:04:16.532274 9226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:16.533702 9226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.60@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:16.538849 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9158
I20260926 13:04:16.542172 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.534418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45180 (local address 127.2.86.62:33835)
0926 13:04:16.534489 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.534491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.534541 (+ 50us) server_negotiation.cc:407] Connection header received
0926 13:04:16.534998 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.534999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.535006 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.535063 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.535616 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.536097 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.537112 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.537289 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.537340 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.539374 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.539376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.539376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.539384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.540772 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.541210 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.541212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.541212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.541275 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.541513 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.541514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.541514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.541694 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.541766 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.542015 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.542087 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:04:16.542382 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.534274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45180)
0926 13:04:16.534431 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.534470 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.534912 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.535133 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.535136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.535359 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.535543 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.535547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.536164 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.536164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.537044 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.537046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.537354 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.537355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.537449 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.537836 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.537844 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.539288 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.540893 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.540899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.540901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.541110 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.541341 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.541341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.541342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.541445 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.541758 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.541763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.541887 (+ 124us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.541993 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.542136 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260926 13:04:16.542949 9226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.60@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
I20260926 13:04:16.547781 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.539334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.2.86.60:40093)
0926 13:04:16.539478 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.539483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.539502 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:04:16.539527 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.539541 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.539552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.539633 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.540422 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.541163 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.541937 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.542399 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.542744 (+ 345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.545143 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.545149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.545149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.545156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.546626 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.546993 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.546996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.546997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.547053 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.547250 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.547250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.547251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.547361 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.547520 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.547597 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.547665 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260926 13:04:16.549387 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.543186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38140)
0926 13:04:16.543232 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.543242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.543277 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.543432 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.543432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.543534 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.543573 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.543574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.544405 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.544406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.545061 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.545063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.545464 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.545464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.545546 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.545784 (+ 238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.545789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.546989 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.548611 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.548612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.548612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.548779 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.548967 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.548967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.548967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.549008 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.549191 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.549192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.549230 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.549279 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.549338 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 13:04:16.549407 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.543264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.2.86.61:35189)
0926 13:04:16.543309 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.543311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.543320 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:16.543346 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.543347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.543360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.543427 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.543641 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.544310 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.545142 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.545394 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.545459 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.547062 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.547063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.547063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.547069 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.548553 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.548833 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.548833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.548834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.548914 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.549052 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.549052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.549052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.549137 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.549180 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.549294 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.549348 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:16.549762 9226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.60@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
I20260926 13:04:16.555163 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.550005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.2.86.60:40093)
0926 13:04:16.550046 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.550048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.550069 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:04:16.550112 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.550113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.550119 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.550160 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.550381 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.550757 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.551306 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.551474 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.551517 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.553065 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.553066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.553066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.553071 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.554199 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.554487 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.554488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.554488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.554539 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.554669 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.554669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.554669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.554785 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.554828 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.554932 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.554995 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 13:04:16.555215 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.549983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40414)
0926 13:04:16.550026 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.550037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.550077 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.550164 (+ 87us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.550165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.550292 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.550322 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.550324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.550809 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.550809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.551247 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.551248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.551528 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.551528 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.551560 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.551935 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.551952 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.553002 (+ 1050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.554263 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.554264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.554264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.554422 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.554592 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.554593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.554593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.554636 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.554833 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.554834 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.554862 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.554918 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.555076 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":49}
I20260926 13:04:16.556362 9226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Bootstrap starting.
I20260926 13:04:16.557643 9226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:16.557847 9226 log.cc:824] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:16.558552 9226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: No bootstrap required, opened a new log
I20260926 13:04:16.559839 9226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } }
I20260926 13:04:16.559959 9226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:16.559981 9226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 851d33f1a0f3460ab83c1efef939d10c, State: Initialized, Role: FOLLOWER
I20260926 13:04:16.560060 9226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } }
I20260926 13:04:16.560277 9231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } } }
I20260926 13:04:16.560328 9226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:16.560349 9231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:04:16.561949 9242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:04:16.562009 9242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:04:16.564028 9158 master_runner.cc:428] Detected that this master 851d33f1a0f3460ab83c1efef939d10c is joining an existing cluster
I20260926 13:04:16.564064 9158 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.86.60:40093
I20260926 13:04:16.571493 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.565165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.2.86.62:33835)
0926 13:04:16.565207 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.565209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.565218 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:16.565256 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.565256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.565265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.565310 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.565666 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.566202 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.566993 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.567203 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.567285 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.568260 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.568261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.568262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.568269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.570319 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.570705 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.570706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.570707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.570766 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.570970 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.570970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.570971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.571096 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.571165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.571329 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.571404 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:04:16.571511 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.564996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45188)
0926 13:04:16.565149 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.565160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.565209 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.565311 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.565312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.565516 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.565591 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.565594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.566316 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.566317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.566907 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.566908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.567296 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.567297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.567356 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.567816 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.567821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.568174 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.570402 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.570403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.570403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.570632 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.570840 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.570840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.570841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.570901 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.571162 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.571162 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.571215 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.571288 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.571383 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260926 13:04:16.572142 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.565038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38146)
0926 13:04:16.565641 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.565651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.565739 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.565914 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.565914 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.566126 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.566178 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.566180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.567092 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.567093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.567946 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.567948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.568423 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.568424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.568501 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.568939 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.568943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.569322 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.571061 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.571062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.571062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.571268 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.571480 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.571480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.571480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.571535 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.571831 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.571832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.571867 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.571969 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.572049 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":77,"threads_started":1}
I20260926 13:04:16.572165 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.565208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.2.86.61:35189)
0926 13:04:16.565252 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.565254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.565715 (+ 461us) server_negotiation.cc:407] Connection header received
0926 13:04:16.565808 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.565809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.565818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.565846 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.566269 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.566996 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.568033 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.568338 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.568423 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.569402 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.569404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.569404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.569410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.571000 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.571346 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.571347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.571348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.571399 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.571613 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.571614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.571615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.571744 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.571786 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.572005 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.572080 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427855, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.62@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
I20260926 13:04:16.592195 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.573825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40434)
0926 13:04:16.573878 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.573889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.573931 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.577088 (+ 3157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.577089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.577237 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.577269 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.577270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.577912 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.577913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.578395 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.578396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.578807 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.578808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.578872 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.579242 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.579246 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.580510 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.591005 (+ 10495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.591006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.591007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.591219 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.591473 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.591474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.591474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.591542 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.591874 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.591875 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.591927 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.592005 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.592102 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:04:16.592540 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.573910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.2.86.60:40093)
0926 13:04:16.576965 (+ 3055us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.576967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.576978 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:04:16.577003 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.577003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.577011 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.577084 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.577343 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.577827 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.578498 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.578723 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.578805 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.580588 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.580589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.580590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.580596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.590902 (+ 10306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.591300 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.591301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.591301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.591371 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.591616 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.591617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.591617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.591795 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.591897 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.592366 (+ 469us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.592441 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3028,"thread_start_us":69,"threads_started":1}
I20260926 13:04:16.593093 8506 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.86.60:40093: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c"
member_type: NON_VOTER
last_known_addr {
host: "127.2.86.60"
port: 40093
}
attrs {
promote: true
}
}
dest_uuid: "35ada61a0f264683b84d554ec77b1949"
cas_config_opid_index: 5
I20260926 13:04:16.593178 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.565483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40426)
0926 13:04:16.571642 (+ 6159us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.571650 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.571696 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.571833 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.571834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.572340 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.572384 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.572386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.573245 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.573246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.573963 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.573965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.574277 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.574278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.574332 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.589801 (+ 15469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.589806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.590174 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.592035 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.592036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.592036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.592189 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.592551 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.592552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.592552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.592610 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.592852 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.592853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.592935 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.593010 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.593091 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6148,"thread_start_us":108,"threads_started":1}
I20260926 13:04:16.593336 8506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } }
I20260926 13:04:16.593397 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.565482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40426 (local address 127.2.86.60:40093)
0926 13:04:16.565517 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.565518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.571678 (+ 6160us) server_negotiation.cc:407] Connection header received
0926 13:04:16.571753 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.571753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.571761 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.571786 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.572462 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.573162 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.574016 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.574202 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.574277 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.590264 (+ 15987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.590264 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.590265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.590271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.591941 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.592431 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.592432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.592432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.592475 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.592678 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.592679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.592679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.592782 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.592842 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.593262 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.593319 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:16.593746 9066 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.594267 9200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:16.594766 9066 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.86.60:40093
I20260926 13:04:16.594833 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.594888 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:16.595115 9200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 13:04:16.595134 9158 master.cc:537] Master@127.2.86.60:40093 shutting down...
I20260926 13:04:16.595424 9066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 13:04:16.596048 8493 consensus_peers.cc:487] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 -> Peer 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60:40093): Couldn't send request to peer 851d33f1a0f3460ab83c1efef939d10c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 13:04:16.598201 9158 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:16.598237 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.593833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38158)
0926 13:04:16.593945 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.593954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.593995 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.594272 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.594273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.594274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:16.594302 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.594303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.595623 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.595624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.596793 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.596796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.597906 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.597907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.597995 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.598038 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.598101 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.598167 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 13:04:16.598306 9158 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:16.598356 9158 tablet_replica.cc:330] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: stopping tablet replica
I20260926 13:04:16.598522 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.593913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.2.86.61:35189)
0926 13:04:16.594124 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.594125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.594133 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:16.594152 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.594153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.594174 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.594226 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:16.594403 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.595129 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.596900 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.597715 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.597801 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.597873 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.598408 (+ 535us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.598444 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260926 13:04:16.598717 9114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 LEARNER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 13:04:16.598908 9066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 13:04:16.599407 9114 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.600664 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ada61a0f264683b84d554ec77b1949. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.600775 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:04:16.600677 9254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: NON_VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: true } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.600950 9254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: LEARNER
I20260926 13:04:16.601308 9252 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: attempting to promote NON_VOTER 25a413f8cd7d48e5890471551cbdfab6 to VOTER
W20260926 13:04:16.601320 8459 connection.cc:535] client connection to 127.2.86.60:40093 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:04:16.601490 9252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } }
I20260926 13:04:16.601989 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.601896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40438)
0926 13:04:16.601923 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.601966 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:40093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260926 13:04:16.602000 9114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 LEARNER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 13:04:16.602154 9252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 13:04:16.602311 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.602198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40454)
0926 13:04:16.602259 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.602286 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:40093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 13:04:16.602381 8493 consensus_peers.cc:487] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 -> Peer 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60:40093): Couldn't send request to peer 851d33f1a0f3460ab83c1efef939d10c. Status: Network error: Client connection negotiation failed: client connection to 127.2.86.60:40093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 13:04:16.602692 9065 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.602846 9114 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.603440 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.603480 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.603523 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:16.603528 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:16.603619 9254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.603694 9254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:04:16.613404 9158 master.cc:552] Master@127.2.86.60:40093 shutdown complete.
I20260926 13:04:16.613472 9158 master_runner.cc:305] Clearing existing system tablet
I20260926 13:04:16.613596 9158 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.613653 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.614250 9158 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.614740 9256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.614847 9158 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.614899 9158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.614925 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "851d33f1a0f3460ab83c1efef939d10c"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "fb4bbbe25516c64db38daa9636a06c0b"
server_key_iv: "350aa0c3e276395de2d0b216e4a8825d"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.615509 9158 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:16.616675 9158 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 13:04:16.616726 9158 log.cc:1196] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 13:04:16.617009 9158 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Deleting consensus metadata
I20260926 13:04:16.617134 9158 master_runner.cc:315] Copying system tablet from 127.2.86.62:33835
I20260926 13:04:16.617712 9158 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: tablet copy: Beginning tablet copy session from remote peer at address 127.2.86.62:33835
I20260926 13:04:16.623597 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.618084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.2.86.62:33835)
0926 13:04:16.618130 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.618132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.618191 (+ 59us) server_negotiation.cc:407] Connection header received
0926 13:04:16.618264 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.618265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.618274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.618328 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.618668 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.619157 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.619844 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.620056 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.620138 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.621145 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.621147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.621147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.621154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.622444 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.622827 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.622828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.622829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.622885 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.623109 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.623110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.623111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.623220 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.623293 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.623437 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.623510 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 13:04:16.623615 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.618032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45196)
0926 13:04:16.618133 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.618143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.618208 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.618333 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.618333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.618536 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.618590 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.618592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.619237 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.619238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.619761 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.619763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.620140 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.620140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.620202 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.620730 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.620734 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.621062 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.622555 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.622556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.622557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.622738 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.622982 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.622983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.622984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.623038 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.623284 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.623285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.623328 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.623414 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.623511 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":62,"threads_started":1}
I20260926 13:04:16.624042 8526 tablet_copy_service.cc:140] P 35ada61a0f264683b84d554ec77b1949: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 851d33f1a0f3460ab83c1efef939d10c ({username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:45196)
I20260926 13:04:16.624100 8526 tablet_copy_service.cc:161] P 35ada61a0f264683b84d554ec77b1949: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 851d33f1a0f3460ab83c1efef939d10c at {username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:45196: session id = 851d33f1a0f3460ab83c1efef939d10c-00000000000000000000000000000000
I20260926 13:04:16.624513 8526 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 13:04:16.624948 9158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 13:04:16.625769 9158 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: tablet copy: Starting download of 0 data blocks...
I20260926 13:04:16.625911 9158 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: tablet copy: Starting download of 1 WAL segments...
I20260926 13:04:16.626613 9158 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 13:04:16.627321 8526 tablet_copy_service.cc:342] P 35ada61a0f264683b84d554ec77b1949: Request end of tablet copy session 851d33f1a0f3460ab83c1efef939d10c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:45196
I20260926 13:04:16.627372 8526 tablet_copy_service.cc:433] P 35ada61a0f264683b84d554ec77b1949: ending tablet copy session 851d33f1a0f3460ab83c1efef939d10c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 851d33f1a0f3460ab83c1efef939d10c
I20260926 13:04:16.629830 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.630977 9263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.631007 9266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.631119 9264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.631342 9158 server_base.cc:1034] running on GCE node
I20260926 13:04:16.631421 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.631560 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.632681 9158 hybrid_clock.cc:648] HybridClock initialized: now 1790427856632675 us; error 24 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.634081 9158 init.cc:377] Logged in from keytab as oryx/127.2.86.60@KRBTEST.COM (short username oryx)
I20260926 13:04:16.634339 9158 webserver.cc:460] Webserver started at http://127.2.86.60:46023/ using document root <none> and password file <none>
I20260926 13:04:16.634457 9158 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.634495 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.635198 9158 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.635742 9273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.635879 9158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:16.635943 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "851d33f1a0f3460ab83c1efef939d10c"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "fb4bbbe25516c64db38daa9636a06c0b"
server_key_iv: "350aa0c3e276395de2d0b216e4a8825d"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.635998 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.651988 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.653760 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.656823 9158 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.60:40093
I20260926 13:04:16.656981 9325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.60:40093 every 8 connection(s)
I20260926 13:04:16.657254 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
I20260926 13:04:16.657943 9326 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:04:16.658352 9326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Bootstrap starting.
I20260926 13:04:16.660650 9326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:04:16.660872 9326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Bootstrap complete.
I20260926 13:04:16.661105 9326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } }
I20260926 13:04:16.661173 9326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 851d33f1a0f3460ab83c1efef939d10c, State: Initialized, Role: LEARNER
I20260926 13:04:16.661199 9326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } }
I20260926 13:04:16.661377 9326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:16.661373 9329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:16.661526 9329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: LEARNER
I20260926 13:04:16.663245 9340 catalog_manager.cc:1259] Loaded cluster ID: 427445596ec04391957d4f3b2741f395
I20260926 13:04:16.663297 9340 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: loading cluster ID for follower catalog manager: success
I20260926 13:04:16.664207 9340 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: acquiring CA information for follower catalog manager: success
I20260926 13:04:16.664475 9340 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:04:16.708817 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.702288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.2.86.60:40093)
0926 13:04:16.702404 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.702406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.702414 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:16.702438 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.702438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.702447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.702520 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.703160 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.703873 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.704901 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.705091 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.705142 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.706497 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.706498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.706498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.706504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.707928 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.708237 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.708238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.708238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.708285 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.708442 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.708443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.708443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.708534 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.708614 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.708700 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.708754 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260926 13:04:16.709375 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8568
I20260926 13:04:16.714290 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:41749
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:33835
--webserver_port=36359
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:16.793555 9342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:16.793758 9342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:16.793780 9342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:16.794950 9342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:16.794991 9342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:16.795042 9342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:16.796464 9342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=36359
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:16.796691 9342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:16.796970 9342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.801692 9348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.801697 9350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.801707 9347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.801959 9342 server_base.cc:1034] running on GCE node
I20260926 13:04:16.802184 9342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.802467 9342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.803645 9342 hybrid_clock.cc:648] HybridClock initialized: now 1790427856803633 us; error 27 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.806264 9342 init.cc:377] Logged in from keytab as oryx/127.2.86.1@KRBTEST.COM (short username oryx)
I20260926 13:04:16.806717 9342 webserver.cc:460] Webserver started at http://127.2.86.1:36359/ using document root <none> and password file <none>
I20260926 13:04:16.806950 9342 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.807030 9342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.808439 9342 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:04:16.809065 9357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.809243 9342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:16.809326 9342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "6bb0b28f0ff130048efc37b92ed88c6d"
server_key_iv: "3b28013416e494c581d8763411e5f8c1"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.809623 9342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.838172 9342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.840076 9342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:16.840222 9342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.840438 9342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:16.840741 9342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:16.840787 9342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.840823 9342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:16.840847 9342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.846753 9342 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41749
I20260926 13:04:16.846805 9470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41749 every 8 connection(s)
I20260926 13:04:16.847149 9342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:16.848431 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9342
I20260926 13:04:16.848601 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8702
I20260926 13:04:16.855346 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:41749
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:33835
--webserver_port=35967
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
I20260926 13:04:16.861109 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57523 (local address 127.2.86.62:33835)
0926 13:04:16.848064 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.848066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.848090 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:04:16.850521 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.850522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.850535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.850565 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.851074 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851831 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853362 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853707 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853748 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.857863 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.857865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.857865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.857871 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.859568 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860016 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.860017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.860017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.860072 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860366 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.860366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.860367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.860493 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.860546 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.860937 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.861017 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 13:04:16.861110 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57551 (local address 127.2.86.60:40093)
0926 13:04:16.848028 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.848030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.848294 (+ 264us) server_negotiation.cc:407] Connection header received
0926 13:04:16.850129 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.850130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.850144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.850185 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.851070 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851814 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853366 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853698 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853743 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.857735 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.857736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.857736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.857744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.859446 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860015 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.860017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.860017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.860072 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860365 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.860366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.860367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.860493 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.860546 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.860937 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.861019 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260926 13:04:16.861640 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:57523)
0926 13:04:16.847962 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.848002 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.850020 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.850624 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.850624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.850786 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.850965 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.850970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851946 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.851947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853268 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.853270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853827 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853972 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.854488 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.854497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.857783 (+ 3286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.859644 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.859644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.859645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.859928 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.860144 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.860145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.860287 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.860582 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.860588 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.860745 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.861255 (+ 510us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.861456 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":232,"thread_start_us":82,"threads_started":1}
I20260926 13:04:16.861668 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.2.86.1:57551)
0926 13:04:16.848236 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.848245 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.850027 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.850265 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.850268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.850696 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.850965 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.850970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851941 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.851941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853268 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.853270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853843 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853972 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.854714 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.854719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.857655 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.859542 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.859548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.859551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.859928 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.860144 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.860145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.860286 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.860582 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.860588 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.860745 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.861190 (+ 445us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.861409 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":62,"thread_start_us":79,"threads_started":1}
I20260926 13:04:16.862112 9471 heartbeater.cc:347] Connected to a master server at 127.2.86.60:40093
I20260926 13:04:16.862112 9472 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:16.862222 9471 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:16.862222 9472 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:16.862432 9471 heartbeater.cc:511] Master 127.2.86.60:40093 requested a full tablet report, sending...
I20260926 13:04:16.862432 9472 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:16.862588 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:36991 (local address 127.2.86.61:35189)
0926 13:04:16.848006 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.848008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.848328 (+ 320us) server_negotiation.cc:407] Connection header received
0926 13:04:16.850178 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.850179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.850191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.850220 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.851070 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851808 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853366 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853646 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853706 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.857585 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.857586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.857587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.857594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.860281 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860701 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.860702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.860702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.860756 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.861009 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.861010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.861010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.861163 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:16.861220 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:16.862381 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0926 13:04:16.862460 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 13:04:16.862808 8506 ts_manager.cc:194] Re-registered known tserver with Master: c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:16.862902 9290 ts_manager.cc:194] Registered new tserver with Master: c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:16.863282 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.847809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.2.86.1:36991)
0926 13:04:16.848238 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.848267 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.850101 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.850281 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.850281 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.850693 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.850965 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.850970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.851941 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.851941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853268 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.853270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.853832 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.853833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.853972 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.854844 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.854848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.857495 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.860381 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.860383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.860609 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.860830 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.860831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.860832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.860918 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.861622 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:16.861623 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:16.861674 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 13:04:16.862932 (+ 1258us) client_negotiation.cc:240] Negotiation successful
0926 13:04:16.863101 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":448,"thread_start_us":55,"threads_started":1}
I20260926 13:04:16.863314 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:57523
I20260926 13:04:16.864030 9473 heartbeater.cc:347] Connected to a master server at 127.2.86.61:35189
I20260926 13:04:16.864090 9473 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:16.864161 9473 heartbeater.cc:511] Master 127.2.86.61:35189 requested a full tablet report, sending...
I20260926 13:04:16.864468 9104 ts_manager.cc:194] Registered new tserver with Master: c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
W20260926 13:04:16.936739 9478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:16.936961 9478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:16.936985 9478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:16.938109 9478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:16.938151 9478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:16.938205 9478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:16.939782 9478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=35967
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:16.940012 9478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:16.940260 9478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:16.945370 9484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.946421 9478 server_base.cc:1034] running on GCE node
W20260926 13:04:16.946895 9483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:16.947036 9486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:16.947235 9478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:16.947446 9478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:16.948581 9478 hybrid_clock.cc:648] HybridClock initialized: now 1790427856948572 us; error 26 us; skew 500 ppm
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:16.951057 9478 init.cc:377] Logged in from keytab as oryx/127.2.86.2@KRBTEST.COM (short username oryx)
I20260926 13:04:16.951470 9478 webserver.cc:460] Webserver started at http://127.2.86.2:35967/ using document root <none> and password file <none>
I20260926 13:04:16.951690 9478 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:16.951771 9478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:16.953173 9478 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:16.953750 9493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.953919 9478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.954021 9478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "b304cd6207334e4e8d01d0f95a14cea3"
format_stamp: "Formatted at 2026-09-26 13:04:15 on dist-test-slave-j4lq"
server_key: "0708a3b7db0005d6b57e21b95304213c"
server_key_iv: "42abb62d7da26605cef497fe3abe99ed"
server_key_version: "encryptionkey@0"
I20260926 13:04:16.954386 9478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:16.979764 9478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:16.981630 9478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:16.981794 9478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:16.982045 9478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:16.982344 9478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:16.982398 9478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.982431 9478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:16.982460 9478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:16.987977 9478 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:41749
I20260926 13:04:16.988030 9606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:41749 every 8 connection(s)
I20260926 13:04:16.988387 9478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:16.990408 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9478
I20260926 13:04:16.990568 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8836
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:16.995963 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:45625
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:33835
--webserver_port=38067
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
Sep 26 13:04:16 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427856, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
I20260926 13:04:17.000624 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51607 (local address 127.2.86.61:35189)
0926 13:04:16.989492 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.989494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.989626 (+ 132us) server_negotiation.cc:407] Connection header received
0926 13:04:16.990144 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.990145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.990157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.990189 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.991820 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.992644 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.993922 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.994213 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.994295 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.996654 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.996655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.996655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.996662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.998937 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999598 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.999599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.999599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.999654 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999920 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.999921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.999921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.000054 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.000095 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.000437 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.000515 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 13:04:17.000669 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:60967 (local address 127.2.86.62:33835)
0926 13:04:16.989398 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.989400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.989604 (+ 204us) server_negotiation.cc:407] Connection header received
0926 13:04:16.990133 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.990134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.990147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.990198 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.991856 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.992456 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.993534 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.993726 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.993808 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.996522 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:16.996524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:16.996524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:16.996531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:16.998234 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999540 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.999541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.999541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:16.999610 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999891 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:16.999891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:16.999892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.000032 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.000075 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.000429 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.000527 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:04:17.001289 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.2.86.2:51607)
0926 13:04:16.989461 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.989500 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.990048 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.990286 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.990288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.991450 (+ 1162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.991716 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.991721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.992740 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.992741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.993796 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.993799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.994292 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.994293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.994356 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.994753 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.994758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.996560 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.999019 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.999027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.999505 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.999725 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.999726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.999825 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.000143 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.000144 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.000283 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.000756 (+ 473us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.000994 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":30,"thread_start_us":83,"threads_started":1}
I20260926 13:04:17.001287 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:60967)
0926 13:04:16.989454 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.989506 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.990023 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.990283 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.990288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.991680 (+ 1392us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.991758 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.991760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.992558 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.992558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.993434 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.993436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.993815 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.993816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.993974 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.994533 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.994542 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:16.996404 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:16.999070 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.999072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.999436 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:16.999684 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:16.999685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:16.999685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:16.999815 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.000119 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.000124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.000281 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.000753 (+ 472us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.001091 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":78,"thread_start_us":110,"threads_started":1}
I20260926 13:04:17.002091 9609 heartbeater.cc:347] Connected to a master server at 127.2.86.61:35189
I20260926 13:04:17.002197 9609 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.002401 9609 heartbeater.cc:511] Master 127.2.86.61:35189 requested a full tablet report, sending...
I20260926 13:04:17.002426 9608 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:17.002477 9608 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.002554 9608 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:17.002856 9104 ts_manager.cc:194] Registered new tserver with Master: b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749)
I20260926 13:04:17.003347 8506 ts_manager.cc:194] Re-registered known tserver with Master: b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749)
I20260926 13:04:17.003727 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:60967
I20260926 13:04:17.005465 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.2.86.2:45529)
0926 13:04:16.989571 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:16.989579 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:16.990048 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:16.990636 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:16.990637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:16.995813 (+ 5176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:16.995883 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.995884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.996994 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.996995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.997938 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:16.997941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.998685 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.998686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:16.998750 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:16.999189 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:16.999194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:17.001231 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:17.003881 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.003881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.003882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.004509 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.004736 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.004736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.004737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.004781 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.005088 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.005088 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.005130 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.005185 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.005264 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":49,"thread_start_us":71,"threads_started":1}
I20260926 13:04:17.005481 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:16.989230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:45529 (local address 127.2.86.60:40093)
0926 13:04:16.989275 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:16.989277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:16.989625 (+ 348us) server_negotiation.cc:407] Connection header received
0926 13:04:16.990145 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:16.990146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:16.990157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:16.990190 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:16.995987 (+ 5797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.996724 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.998044 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:16.998421 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:16.998456 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.001334 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.001336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.001336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.001345 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.003256 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.004600 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.004601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.004601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.004668 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.004847 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.004849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.004849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.005015 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.005101 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.005252 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.005340 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:17.005817 9607 heartbeater.cc:347] Connected to a master server at 127.2.86.60:40093
I20260926 13:04:17.005874 9607 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.005954 9607 heartbeater.cc:511] Master 127.2.86.60:40093 requested a full tablet report, sending...
I20260926 13:04:17.006289 9290 ts_manager.cc:194] Registered new tserver with Master: b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749)
W20260926 13:04:17.078003 9614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:17.078186 9614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:17.078208 9614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:17.079241 9614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:17.079273 9614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:17.079298 9614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:17.080649 9614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:45625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=38067
--tserver_master_addrs=127.2.86.62:33835,127.2.86.61:35189,127.2.86.60:40093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:17.080824 9614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:17.081079 9614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:17.085740 9622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:17.085752 9620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:17.085731 9619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:17.086061 9614 server_base.cc:1034] running on GCE node
I20260926 13:04:17.086252 9614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:17.086468 9614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:17.087643 9614 hybrid_clock.cc:648] HybridClock initialized: now 1790427857087637 us; error 29 us; skew 500 ppm
Sep 26 13:04:17 dist-test-slave-j4lq krb5kdc[8461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427857, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:17.090093 9614 init.cc:377] Logged in from keytab as oryx/127.2.86.3@KRBTEST.COM (short username oryx)
I20260926 13:04:17.090533 9614 webserver.cc:460] Webserver started at http://127.2.86.3:38067/ using document root <none> and password file <none>
I20260926 13:04:17.090739 9614 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:17.090787 9614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:17.091975 9614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:17.092481 9629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:17.092638 9614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:17.092715 9614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "763bb84a9d2b4a828a54b3a5f7b3358b"
format_stamp: "Formatted at 2026-09-26 13:04:16 on dist-test-slave-j4lq"
server_key: "b2e0f9169bfa33a9f7597189d1ed74c0"
server_key_iv: "50ec967310ce9da37958d2e98ccb09c5"
server_key_version: "encryptionkey@0"
I20260926 13:04:17.093057 9614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:17.111749 9614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:17.114116 9614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:17.114221 9614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:17.114400 9614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:17.114729 9614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:17.114763 9614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:17.114807 9614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:17.114832 9614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:17.120559 9614 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:45625
I20260926 13:04:17.120625 9742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:45625 every 8 connection(s)
I20260926 13:04:17.120982 9614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:17.122969 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9614
Sep 26 13:04:17 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427857, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.60@KRBTEST.COM
Sep 26 13:04:17 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427857, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
Sep 26 13:04:17 dist-test-slave-j4lq krb5kdc[8461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427857, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.61@KRBTEST.COM
I20260926 13:04:17.132661 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:52425 (local address 127.2.86.60:40093)
0926 13:04:17.121771 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.121773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.121976 (+ 203us) server_negotiation.cc:407] Connection header received
0926 13:04:17.122599 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.122601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.122612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.122662 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.123719 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124292 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125493 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125765 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125812 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.129179 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.129180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.129181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.129187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.130837 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.131530 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.131531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.131532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.131585 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.131908 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.131910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.131910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.132079 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.132152 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.132472 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.132544 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:04:17.133184 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.2.86.3:52425)
0926 13:04:17.121865 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.121902 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.122504 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.123173 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.123173 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.123421 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:17.123599 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.123607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124394 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.124395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.125384 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.125387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125895 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.126036 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.126754 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:17.126763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:17.129094 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:17.131141 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.131153 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.131155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.131452 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.131652 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.131653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.131653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.131774 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.132145 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.132151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.132303 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.132783 (+ 480us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.132967 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260926 13:04:17.133682 9743 heartbeater.cc:347] Connected to a master server at 127.2.86.60:40093
I20260926 13:04:17.133777 9743 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.133975 9743 heartbeater.cc:511] Master 127.2.86.60:40093 requested a full tablet report, sending...
I20260926 13:04:17.134397 9290 ts_manager.cc:194] Registered new tserver with Master: 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3:45625)
I20260926 13:04:17.134420 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:37545 (local address 127.2.86.62:33835)
0926 13:04:17.121819 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.121821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.122216 (+ 395us) server_negotiation.cc:407] Connection header received
0926 13:04:17.122618 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.122619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.122631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.122663 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.123708 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124464 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125568 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125851 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125888 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.130225 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.130226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.130226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.130234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.133092 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133528 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.133529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.133529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.133584 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133784 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.133785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.133785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.133914 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.133977 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.134205 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.134286 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:04:17.134753 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:37545)
0926 13:04:17.122114 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.122123 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.122505 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.122760 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.122775 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.123307 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:17.123598 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.123605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124558 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.124559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.125477 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.125478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125963 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.126036 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.126862 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:17.126868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:17.129389 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:17.133180 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.133181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.133422 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.133660 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133660 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.133661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.133714 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.134026 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.134027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.134068 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.134528 (+ 460us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.134622 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":366,"thread_start_us":75,"threads_started":1}
I20260926 13:04:17.135246 9744 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33835
I20260926 13:04:17.135296 9744 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.135355 9744 heartbeater.cc:511] Master 127.2.86.62:33835 requested a full tablet report, sending...
I20260926 13:04:17.135599 8506 ts_manager.cc:194] Re-registered known tserver with Master: 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3:45625)
I20260926 13:04:17.136062 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:46889 (local address 127.2.86.61:35189)
0926 13:04:17.122021 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.122023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.122033 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:17.122618 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.122619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.122631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.122663 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.123708 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124472 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125568 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125851 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125888 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.131871 (+ 5983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.131872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.131872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.131879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.133514 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133890 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.133891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.133892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.133965 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.135028 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.135029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.135029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.135160 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.135206 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.135847 (+ 641us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.135935 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:04:17.136101 8506 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:37545
I20260926 13:04:17.137293 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.121696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.2.86.3:46889)
0926 13:04:17.121862 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.121901 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.122502 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.122760 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.122775 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.123327 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:17.123598 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.123605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.124558 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.124559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.125472 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.125474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.125965 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.125966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.126036 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.126862 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:17.126868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:17.131782 (+ 4914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:17.133598 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.133599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.133599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.133804 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.134043 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:17.134043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:17.134044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:17.134097 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:17.135646 (+ 1549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:17.135648 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:17.135690 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.137032 (+ 1342us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.137146 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":34,"thread_start_us":87,"threads_started":1}
I20260926 13:04:17.137602 9745 heartbeater.cc:347] Connected to a master server at 127.2.86.61:35189
I20260926 13:04:17.137656 9745 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:17.137724 9745 heartbeater.cc:511] Master 127.2.86.61:35189 requested a full tablet report, sending...
I20260926 13:04:17.137991 9104 ts_manager.cc:194] Registered new tserver with Master: 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3:45625)
I20260926 13:04:17.138334 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.125131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.2.86.60:40093)
0926 13:04:17.126130 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.126131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.126617 (+ 486us) server_negotiation.cc:407] Connection header received
0926 13:04:17.126636 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.126637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.126648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.126699 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.128745 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.129454 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.130463 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.130775 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.130835 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.134722 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.134724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.134724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.134730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.136936 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.137319 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.137321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.137321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.137376 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.137580 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.137580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.137581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.137702 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.137793 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.138149 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.138232 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":55,"threads_started":1}
I20260926 13:04:17.138738 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.125360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.2.86.62:33835)
0926 13:04:17.126181 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.126183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.126291 (+ 108us) server_negotiation.cc:407] Connection header received
0926 13:04:17.126366 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.126367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.126378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.126454 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.128632 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.129721 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.131171 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.131504 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.131562 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.133612 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.133613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.133614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.133620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.136709 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.137220 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.137221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.137222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.137280 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.137796 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.137797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.137798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.137935 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.138015 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.138365 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.138647 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":48,"threads_started":1}
I20260926 13:04:17.139163 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.125658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.2.86.61:35189)
0926 13:04:17.126278 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.126280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.126438 (+ 158us) server_negotiation.cc:407] Connection header received
0926 13:04:17.126489 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.126490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.126503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.126623 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.127590 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.128050 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.129431 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.130108 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.130166 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.136055 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.136056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.136057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.136062 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.137920 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.138408 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.138409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.138409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.138483 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.138685 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.138686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.138687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.138807 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.138912 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.139013 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.139084 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":46,"threads_started":1}
I20260926 13:04:17.140084 8506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:17.141772 8506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:17.148995 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.2.86.3:45625)
0926 13:04:17.145811 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.145815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.145823 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:17.145862 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.145864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.145877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.145945 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.146113 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.146708 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.147959 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.148628 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148713 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.148777 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.148900 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.148933 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":85,"threads_started":1}
I20260926 13:04:17.148998 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:45625 (local address 127.0.0.1:43690)
0926 13:04:17.145170 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.145179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.145229 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.146003 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.146004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.146006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.146041 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.146042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.146787 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.146788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.147878 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.147880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.148731 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.148794 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.148798 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.148846 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.148932 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":71,"threads_started":1}
I20260926 13:04:17.149353 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.2.86.2:41749)
0926 13:04:17.145462 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.145466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.145565 (+ 99us) server_negotiation.cc:407] Connection header received
0926 13:04:17.145638 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.145641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.145659 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.145736 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.145960 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.147026 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148218 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.148707 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148783 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.148855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.149234 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.149279 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260926 13:04:17.149386 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41749 (local address 127.0.0.1:42300)
0926 13:04:17.145653 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.145662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.145706 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.145952 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.145953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.145956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.145997 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.145999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.147171 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.147172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.148181 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.148183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.149183 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.149183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.149241 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.149245 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.149290 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.149326 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":90,"threads_started":1}
I20260926 13:04:17.149555 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:41749 (local address 127.0.0.1:60316)
0926 13:04:17.145484 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.145500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.145570 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.145805 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.145806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.145808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.145860 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.145861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.147109 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.147110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.148144 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.148145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.148900 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.148997 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.149000 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.149455 (+ 455us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.149496 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":90,"threads_started":1}
I20260926 13:04:17.149631 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.145297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.2.86.1:41749)
0926 13:04:17.145529 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.145532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.145703 (+ 171us) server_negotiation.cc:407] Connection header received
0926 13:04:17.145777 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.145779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.145794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.145878 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.146105 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.147114 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.148266 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.149013 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.149078 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.149152 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.149498 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.149550 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260926 13:04:17.149683 9677 tablet_service.cc:1467] Processing CreateTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a (DEFAULT_TABLE table=test-table [id=44536822d79f4747a5ad7a5794b6640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:17.150070 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f41c2f58644485be58e6a70a4a5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:17.150202 9541 tablet_service.cc:1467] Processing CreateTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a (DEFAULT_TABLE table=test-table [id=44536822d79f4747a5ad7a5794b6640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:17.150501 9541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f41c2f58644485be58e6a70a4a5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:17.153486 9772 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Bootstrap starting.
I20260926 13:04:17.154002 9405 tablet_service.cc:1467] Processing CreateTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a (DEFAULT_TABLE table=test-table [id=44536822d79f4747a5ad7a5794b6640e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:17.154264 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f41c2f58644485be58e6a70a4a5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:17.154476 9772 tablet_bootstrap.cc:654] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:17.154742 9772 log.cc:824] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:17.155442 9772 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: No bootstrap required, opened a new log
I20260926 13:04:17.155520 9772 ts_tablet_manager.cc:1397] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:17.157160 9775 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Bootstrap starting.
I20260926 13:04:17.157279 9773 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Bootstrap starting.
I20260926 13:04:17.157150 9772 raft_consensus.cc:348] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.157379 9772 raft_consensus.cc:374] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:17.157433 9772 raft_consensus.cc:729] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763bb84a9d2b4a828a54b3a5f7b3358b, State: Initialized, Role: FOLLOWER
I20260926 13:04:17.157550 9772 consensus_queue.cc:260] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.157747 9775 tablet_bootstrap.cc:654] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:17.157801 9773 tablet_bootstrap.cc:654] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:17.157809 9772 ts_tablet_manager.cc:1428] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:04:17.157950 9744 heartbeater.cc:503] Master 127.2.86.62:33835 was elected leader, sending a full tablet report...
I20260926 13:04:17.158298 9775 log.cc:824] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:17.158581 9773 log.cc:824] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:17.159317 9775 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: No bootstrap required, opened a new log
I20260926 13:04:17.159386 9775 ts_tablet_manager.cc:1397] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:17.159574 9773 tablet_bootstrap.cc:492] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: No bootstrap required, opened a new log
I20260926 13:04:17.159626 9773 ts_tablet_manager.cc:1397] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:17.161024 9775 raft_consensus.cc:348] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.161181 9775 raft_consensus.cc:374] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:17.161224 9775 raft_consensus.cc:729] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ecf03c9d5d4cd78c391b46c8126ebb, State: Initialized, Role: FOLLOWER
I20260926 13:04:17.161327 9775 consensus_queue.cc:260] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.161615 9775 ts_tablet_manager.cc:1428] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:17.161790 9472 heartbeater.cc:503] Master 127.2.86.62:33835 was elected leader, sending a full tablet report...
I20260926 13:04:17.161903 9773 raft_consensus.cc:348] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.162045 9773 raft_consensus.cc:374] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:17.162089 9773 raft_consensus.cc:729] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b304cd6207334e4e8d01d0f95a14cea3, State: Initialized, Role: FOLLOWER
I20260926 13:04:17.162189 9773 consensus_queue.cc:260] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.162422 9773 ts_tablet_manager.cc:1428] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 13:04:17.162762 9608 heartbeater.cc:503] Master 127.2.86.62:33835 was elected leader, sending a full tablet report...
I20260926 13:04:17.166088 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.163120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40486)
0926 13:04:17.163162 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.163173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.163218 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.163500 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.163502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.163504 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.163544 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.163546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.164331 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.164331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.165188 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.165190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.165868 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.165869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.165933 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.165936 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.165994 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.166033 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 13:04:17.166087 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.163291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.2.86.60:40093)
0926 13:04:17.163363 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.163365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.163384 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:04:17.163415 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.163416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.163435 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.163496 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.163603 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.164266 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.165277 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.165761 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.165845 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.165906 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.166000 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.166043 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:04:17.166352 9300 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260926 13:04:17.166958 9781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ada61a0f264683b84d554ec77b1949. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:17.167064 9781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: LEARNER
I20260926 13:04:17.169314 9776 raft_consensus.cc:482] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:17.169377 9776 raft_consensus.cc:504] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.169718 9776 leader_election.cc:288] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749), c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:17.172633 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.169877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34705 (local address 127.2.86.2:41749)
0926 13:04:17.169930 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.169932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.170048 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:04:17.170065 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.170066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.170073 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.170109 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.170275 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.170733 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.171644 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.172316 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.172395 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.172404 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.172563 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.172592 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 13:04:17.172636 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.169846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:41749 (local address 127.2.86.3:34705)
0926 13:04:17.169945 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.169955 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.169997 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.170131 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.170132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.170134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.170207 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.170208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.170815 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.170816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.171551 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.171552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.172406 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.172406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.172479 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.172482 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.172540 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.172573 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":37}
I20260926 13:04:17.173005 9561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f41c2f58644485be58e6a70a4a5f6a" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b304cd6207334e4e8d01d0f95a14cea3" is_pre_election: true
I20260926 13:04:17.173136 9561 raft_consensus.cc:2454] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 0.
I20260926 13:04:17.173343 9630 leader_election.cc:302] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763bb84a9d2b4a828a54b3a5f7b3358b, b304cd6207334e4e8d01d0f95a14cea3; no voters:
I20260926 13:04:17.173447 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.169842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41749 (local address 127.2.86.3:38511)
0926 13:04:17.169932 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:17.169940 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:17.169981 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:17.170157 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:17.170158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:17.170159 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:17.170235 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.170270 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.170859 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.170860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.172417 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:17.172420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.173128 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.173129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:17.173210 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.173214 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:17.173268 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:04:17.173308 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 13:04:17.173496 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.169973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38511 (local address 127.2.86.1:41749)
0926 13:04:17.170030 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.170032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.170040 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:17.170060 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.170060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.170069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.170142 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:17.170344 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.170750 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.172500 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.173019 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.173088 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.173098 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.173387 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.173416 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:04:17.173576 9776 raft_consensus.cc:2781] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:17.173633 9776 raft_consensus.cc:482] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:17.173667 9776 raft_consensus.cc:3037] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:17.173727 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f41c2f58644485be58e6a70a4a5f6a" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" is_pre_election: true
I20260926 13:04:17.173864 9425 raft_consensus.cc:2454] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 0.
I20260926 13:04:17.174427 9776 raft_consensus.cc:504] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.174644 9776 leader_election.cc:288] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 election: Requested vote from peers b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749), c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:17.174676 9561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f41c2f58644485be58e6a70a4a5f6a" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b304cd6207334e4e8d01d0f95a14cea3"
I20260926 13:04:17.174736 9561 raft_consensus.cc:3037] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:17.175349 9561 raft_consensus.cc:2454] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 1.
I20260926 13:04:17.175518 9630 leader_election.cc:302] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763bb84a9d2b4a828a54b3a5f7b3358b, b304cd6207334e4e8d01d0f95a14cea3; no voters:
I20260926 13:04:17.175539 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f41c2f58644485be58e6a70a4a5f6a" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
I20260926 13:04:17.175613 9776 raft_consensus.cc:2781] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:17.175611 9425 raft_consensus.cc:3037] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:17.175729 9776 raft_consensus.cc:686] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 LEADER]: Becoming Leader. State: Replica: 763bb84a9d2b4a828a54b3a5f7b3358b, State: Running, Role: LEADER
I20260926 13:04:17.175789 9776 consensus_queue.cc:237] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } }
I20260926 13:04:17.176398 9425 raft_consensus.cc:2454] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 1.
I20260926 13:04:17.176733 8506 catalog_manager.cc:5495] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b reported cstate change: term changed from 0 to 1, leader changed from <none> to 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3). New cstate: current_term: 1 leader_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:17.177237 9761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: NON_VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: true } } }
I20260926 13:04:17.177351 9761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:17.177742 9761 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: attempting to promote NON_VOTER 851d33f1a0f3460ab83c1efef939d10c to VOTER
I20260926 13:04:17.177865 9761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } }
I20260926 13:04:17.178299 9300 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 LEARNER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 13:04:17.178476 9789 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260926 13:04:17.178638 9114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Refusing update from remote peer 35ada61a0f264683b84d554ec77b1949: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 13:04:17.178797 9789 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 13:04:17.179500 9300 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.179616 9114 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.180438 9781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.180529 9781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:04:17.179432 9790 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.180964 9791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.181044 9791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:04:17.181427 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.181514 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:17.181449 9790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.181452 9761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35ada61a0f264683b84d554ec77b1949" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:17.181543 9790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:17.181558 9761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:17.185365 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.183299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.2.86.3:45625)
0926 13:04:17.183330 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.183332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.183338 (+ 6us) server_negotiation.cc:407] Connection header received
0926 13:04:17.183359 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.183360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.183372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.183430 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:17.183552 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.183973 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.184655 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.184820 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.184924 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.185050 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:17.185125 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:17.185178 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.185310 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.185323 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:04:17.186767 9425 raft_consensus.cc:1261] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Refusing update from remote peer 763bb84a9d2b4a828a54b3a5f7b3358b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:17.186812 9561 raft_consensus.cc:1261] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Refusing update from remote peer 763bb84a9d2b4a828a54b3a5f7b3358b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:17.187117 9776 consensus_queue.cc:1035] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:17.187242 9786 consensus_queue.cc:1035] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:17.190292 9795 mvcc.cc:205] Tried to move back new op lower bound from 7333592503034847232 to 7333592502992695296. Current Snapshot: MvccSnapshot[applied={T|T < 7333592503034847232}]
I20260926 13:04:17.190712 9794 mvcc.cc:205] Tried to move back new op lower bound from 7333592503034847232 to 7333592502992695296. Current Snapshot: MvccSnapshot[applied={T|T < 7333592503034847232}]
I20260926 13:04:17.190932 9793 mvcc.cc:205] Tried to move back new op lower bound from 7333592503034847232 to 7333592502992695296. Current Snapshot: MvccSnapshot[applied={T|T < 7333592503034847232}]
I20260926 13:04:17.192926 8506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:17.193004 8506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:17.194377 8506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35ada61a0f264683b84d554ec77b1949: Sending DeleteTablet for 3 replicas of tablet e1f41c2f58644485be58e6a70a4a5f6a
I20260926 13:04:17.194629 9541 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:17 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:60316
I20260926 13:04:17.194630 9677 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:17 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:43690
I20260926 13:04:17.194695 9405 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f41c2f58644485be58e6a70a4a5f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:17 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42300
I20260926 13:04:17.194862 9803 tablet_replica.cc:330] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: stopping tablet replica
I20260926 13:04:17.194983 9803 raft_consensus.cc:2229] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:17.195120 9803 raft_consensus.cc:2258] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:17.195525 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8474
I20260926 13:04:17.195578 9803 ts_tablet_manager.cc:1905] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:17.196985 9802 tablet_replica.cc:330] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: stopping tablet replica
I20260926 13:04:17.197077 9802 raft_consensus.cc:2229] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:17.197183 9802 raft_consensus.cc:2258] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:17.197580 9802 ts_tablet_manager.cc:1905] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:17.198460 9803 ts_tablet_manager.cc:1918] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:17.198575 9803 log.cc:1196] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/e1f41c2f58644485be58e6a70a4a5f6a
I20260926 13:04:17.198894 9803 ts_tablet_manager.cc:1939] T e1f41c2f58644485be58e6a70a4a5f6a P b304cd6207334e4e8d01d0f95a14cea3: Deleting consensus metadata
I20260926 13:04:17.199239 9802 ts_tablet_manager.cc:1918] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:17.199311 9802 log.cc:1196] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/e1f41c2f58644485be58e6a70a4a5f6a
I20260926 13:04:17.199561 9802 ts_tablet_manager.cc:1939] T e1f41c2f58644485be58e6a70a4a5f6a P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting consensus metadata
W20260926 13:04:17.201535 9631 connection.cc:535] server connection from 127.0.0.1:43690 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 13:04:17.201855 9495 connection.cc:535] server connection from 127.0.0.1:60316 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:04:17.204978 9804 tablet_replica.cc:330] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: stopping tablet replica
I20260926 13:04:17.205086 9804 raft_consensus.cc:2229] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:17.205189 9804 raft_consensus.cc:2258] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:17.210584 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.202826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.2.86.61:35189)
0926 13:04:17.202869 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.202870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.203198 (+ 328us) server_negotiation.cc:407] Connection header received
0926 13:04:17.203255 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.203256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.203267 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.203324 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.203867 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.204657 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.205601 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.205794 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.205873 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.207132 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.207134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.207134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.207141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.208976 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.209447 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.209448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.209449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.209502 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.209707 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.209708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.209709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.209859 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.209962 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.210439 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.210521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:04:17.212167 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:17.203039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.2.86.60:40093)
0926 13:04:17.203082 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:17.203084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:17.205085 (+ 2001us) server_negotiation.cc:407] Connection header received
0926 13:04:17.205109 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:17.205110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:17.205120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:17.205176 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:17.205849 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.206561 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.207590 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:17.207924 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:17.207987 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:17.209416 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:17.209417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:17.209418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:17.209425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:17.211059 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.211450 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.211451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.211451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.211507 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:17.211705 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:17.211706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:17.211706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:17.211839 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:17.211946 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:17.212057 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 13:04:17.212113 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:04:17.212600 9804 ts_tablet_manager.cc:1905] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:17.213923 9804 ts_tablet_manager.cc:1918] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:17.214015 9804 log.cc:1196] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/e1f41c2f58644485be58e6a70a4a5f6a
I20260926 13:04:17.214306 9804 ts_tablet_manager.cc:1939] T e1f41c2f58644485be58e6a70a4a5f6a P c8ecf03c9d5d4cd78c391b46c8126ebb: Deleting consensus metadata
W20260926 13:04:17.214638 9361 connection.cc:441] server connection from 127.0.0.1:42300 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42300 (request call id 1) could send its response
W20260926 13:04:17.849090 9466 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.2.86.1:41749
W20260926 13:04:17.989764 9602 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.2.86.2:41749
I20260926 13:04:18.191001 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.190683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:46591)
0926 13:04:18.190791 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.190870 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
I20260926 13:04:18.191470 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.191085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:49855)
0926 13:04:18.191370 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.191430 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":45,"threads_started":1}
W20260926 13:04:18.191692 9472 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33835 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
W20260926 13:04:18.192112 9608 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33835 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
I20260926 13:04:18.192284 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.191434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:39171)
0926 13:04:18.192193 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.192248 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":56,"threads_started":1}
W20260926 13:04:18.192410 9744 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33835 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
I20260926 13:04:18.695281 9815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35ada61a0f264683b84d554ec77b1949)
I20260926 13:04:18.695289 9816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 35ada61a0f264683b84d554ec77b1949)
I20260926 13:04:18.695386 9815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } }
I20260926 13:04:18.695416 9816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } }
I20260926 13:04:18.695834 9816 leader_election.cc:288] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835), 851d33f1a0f3460ab83c1efef939d10c (127.2.86.60:40093)
I20260926 13:04:18.695885 9815 leader_election.cc:288] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835), 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61:35189)
I20260926 13:04:18.695971 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.695744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45646)
0926 13:04:18.695838 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.695879 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260926 13:04:18.696020 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.695743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45648)
0926 13:04:18.695948 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.695986 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":44,"threads_started":1}
W20260926 13:04:18.696113 9275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
W20260926 13:04:18.696132 9089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
I20260926 13:04:18.696576 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.696470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45662)
0926 13:04:18.696501 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.696535 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260926 13:04:18.696635 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.696550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45670)
0926 13:04:18.696587 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.696609 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260926 13:04:18.696676 9089 leader_election.cc:334] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835): Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
W20260926 13:04:18.696708 9275 leader_election.cc:334] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835): Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
I20260926 13:04:18.698846 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.695805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:35189 (local address 127.0.0.1:38184)
0926 13:04:18.696145 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.696155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.696206 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.696390 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:18.696391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:18.696393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:18.696449 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.696451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.697349 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.697350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.698084 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.698086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.698612 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.698672 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.698675 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:18.698745 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:04:18.698783 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":50,"threads_started":1}
I20260926 13:04:18.698849 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.695843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.2.86.61:35189)
0926 13:04:18.696259 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.696260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.696268 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:18.696313 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.696314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.696324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.696377 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:18.696526 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.697274 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698183 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.698534 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698613 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.698624 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.698753 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.698786 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":37,"threads_started":1}
I20260926 13:04:18.699061 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.695969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:40093 (local address 127.0.0.1:40512)
0926 13:04:18.696328 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.696339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.696506 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.696933 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:18.696934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:18.696936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:18.696974 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.696975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.697572 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.697573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.698308 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.698310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.698827 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.698889 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.698900 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 13:04:18.698954 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:04:18.699002 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":32,"threads_started":1}
I20260926 13:04:18.699075 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.696080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.2.86.60:40093)
0926 13:04:18.696172 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.696173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.696422 (+ 249us) server_negotiation.cc:407] Connection header received
0926 13:04:18.696586 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.696587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.696597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.696921 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:18.697042 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.697489 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698402 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.698738 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.698826 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.698835 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.698986 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.699028 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":30,"threads_started":1}
I20260926 13:04:18.699067 9114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "851d33f1a0f3460ab83c1efef939d10c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "25a413f8cd7d48e5890471551cbdfab6" is_pre_election: true
I20260926 13:04:18.699177 9114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 851d33f1a0f3460ab83c1efef939d10c in term 1.
I20260926 13:04:18.699366 9274 leader_election.cc:302] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25a413f8cd7d48e5890471551cbdfab6, 851d33f1a0f3460ab83c1efef939d10c; no voters: 35ada61a0f264683b84d554ec77b1949
I20260926 13:04:18.699477 9815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 13:04:18.699517 9815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 35ada61a0f264683b84d554ec77b1949)
I20260926 13:04:18.699536 9815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:04:18.699571 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25a413f8cd7d48e5890471551cbdfab6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "851d33f1a0f3460ab83c1efef939d10c" is_pre_election: true
I20260926 13:04:18.700241 9815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } }
I20260926 13:04:18.700359 9300 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25a413f8cd7d48e5890471551cbdfab6 in current term 2: Already voted for candidate 851d33f1a0f3460ab83c1efef939d10c in this term.
I20260926 13:04:18.700384 9815 leader_election.cc:288] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 election: Requested vote from peers 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835), 25a413f8cd7d48e5890471551cbdfab6 (127.2.86.61:35189)
I20260926 13:04:18.700521 9114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "851d33f1a0f3460ab83c1efef939d10c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "25a413f8cd7d48e5890471551cbdfab6"
I20260926 13:04:18.700551 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.700449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45680)
0926 13:04:18.700500 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.700530 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 13:04:18.700558 9091 leader_election.cc:302] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a413f8cd7d48e5890471551cbdfab6; no voters: 35ada61a0f264683b84d554ec77b1949, 851d33f1a0f3460ab83c1efef939d10c
I20260926 13:04:18.700590 9114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:04:18.701259 9114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 851d33f1a0f3460ab83c1efef939d10c in term 2.
I20260926 13:04:18.701340 9816 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:04:18.701431 9274 leader_election.cc:302] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a413f8cd7d48e5890471551cbdfab6, 851d33f1a0f3460ab83c1efef939d10c; no voters:
I20260926 13:04:18.701531 9815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:04:18.701593 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.701512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45696)
0926 13:04:18.701548 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.701568 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 13:04:18.701644 9275 leader_election.cc:334] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835): Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
I20260926 13:04:18.701673 9815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [term 2 LEADER]: Becoming Leader. State: Replica: 851d33f1a0f3460ab83c1efef939d10c, State: Running, Role: LEADER
I20260926 13:04:18.701738 9815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } }
I20260926 13:04:18.702062 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 851d33f1a0f3460ab83c1efef939d10c. Latest consensus state: current_term: 2 leader_uuid: "851d33f1a0f3460ab83c1efef939d10c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:18.702147 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:18.702302 9827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:18.702723 9827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:18.702796 9827 catalog_manager.cc:1259] Loaded cluster ID: 427445596ec04391957d4f3b2741f395
I20260926 13:04:18.702831 9827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:18.702970 9827 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:18.703049 9827 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Loaded TSK: 0
I20260926 13:04:18.703198 9827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:18.745849 9290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:18.747387 9290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:18.748333 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.748230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45712)
0926 13:04:18.748279 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.748311 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 13:04:18.748337 9114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [term 2 FOLLOWER]: Refusing update from remote peer 851d33f1a0f3460ab83c1efef939d10c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 13:04:18.748636 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.748534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.0.0.1:45718)
0926 13:04:18.748576 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.748611 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:04:18.748579 9825 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 13:04:18.748739 9275 consensus_peers.cc:487] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c -> Peer 35ada61a0f264683b84d554ec77b1949 (127.2.86.62:33835): Couldn't send request to peer 35ada61a0f264683b84d554ec77b1949. Status: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 13:04:18.749187 9816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 851d33f1a0f3460ab83c1efef939d10c. Latest consensus state: current_term: 2 leader_uuid: "851d33f1a0f3460ab83c1efef939d10c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:18.749279 9816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:04:18.749657 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "851d33f1a0f3460ab83c1efef939d10c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:18.749742 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:18.750094 9816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "851d33f1a0f3460ab83c1efef939d10c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "35ada61a0f264683b84d554ec77b1949" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33835 } } peers { permanent_uuid: "25a413f8cd7d48e5890471551cbdfab6" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 35189 } attrs { promote: false } } peers { permanent_uuid: "851d33f1a0f3460ab83c1efef939d10c" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 40093 } attrs { promote: false } } }
I20260926 13:04:18.750173 9816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a413f8cd7d48e5890471551cbdfab6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:04:18.750416 9829 mvcc.cc:205] Tried to move back new op lower bound from 7333592509431422976 to 7333592509242970112. Current Snapshot: MvccSnapshot[applied={T|T < 7333592509431422976 or (T in {7333592509431422976})}]
I20260926 13:04:18.754602 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:41749 (local address 127.0.0.1:60326)
0926 13:04:18.751398 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.751408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.751446 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.751866 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:18.751866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:18.751868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:18.751918 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.751919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752616 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.752617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.753697 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.753699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.754369 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.754428 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.754432 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:18.754507 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 13:04:18.754544 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 13:04:18.754624 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.2.86.2:41749)
0926 13:04:18.751584 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.751586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.751593 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:18.751617 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.751617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.751634 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.751979 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:18.752018 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752537 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.753786 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.754279 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754370 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.754382 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.754515 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.754554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260926 13:04:18.754750 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41749 (local address 127.0.0.1:42310)
0926 13:04:18.751496 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.751506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.751552 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.751976 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:18.751976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:18.751979 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:18.752023 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.752025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752935 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.752936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.753726 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.753729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.754474 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.754536 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.754540 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:18.754568 (+ 28us) client_negotiation.cc:240] Negotiation successful
0926 13:04:18.754610 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 13:04:18.754884 9541 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 (DEFAULT_TABLE table=test-table [id=ebc8e4d1cec64820bf50d4f09453b188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:18.755018 9541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea0ed1e16694b0aa2f1a9221741f557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:18.755251 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.2.86.3:45625)
0926 13:04:18.751816 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.751818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.751826 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:18.751849 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.751849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.751860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.751913 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:18.752056 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752795 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754015 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.754564 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754623 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.754636 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.755162 (+ 526us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.755201 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
I20260926 13:04:18.755254 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:45625 (local address 127.0.0.1:43716)
0926 13:04:18.751735 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.751746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.751785 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.751964 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:18.751965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:18.751966 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:18.751994 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.751995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752893 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.752894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.753938 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:18.753940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.755035 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.755036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:18.755092 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.755097 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:04:18.755157 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 13:04:18.755197 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":70,"threads_started":1}
I20260926 13:04:18.755470 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 (DEFAULT_TABLE table=test-table [id=ebc8e4d1cec64820bf50d4f09453b188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:18.755585 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea0ed1e16694b0aa2f1a9221741f557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:18.756495 9838 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Bootstrap starting.
I20260926 13:04:18.756640 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.751702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.2.86.1:41749)
0926 13:04:18.751860 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.751862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.751871 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:18.751899 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.751900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.751912 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.751971 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:18.752098 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.752795 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.753821 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.754361 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.754653 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.754667 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.756502 (+ 1835us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.756551 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":54,"threads_started":1}
I20260926 13:04:18.756776 9839 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Bootstrap starting.
I20260926 13:04:18.756891 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 (DEFAULT_TABLE table=test-table [id=ebc8e4d1cec64820bf50d4f09453b188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:18.757019 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea0ed1e16694b0aa2f1a9221741f557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:18.757193 9839 tablet_bootstrap.cc:654] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:18.757774 9839 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: No bootstrap required, opened a new log
I20260926 13:04:18.757833 9839 ts_tablet_manager.cc:1397] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:18.757947 9839 raft_consensus.cc:348] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.758018 9839 raft_consensus.cc:374] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:18.758035 9839 raft_consensus.cc:729] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763bb84a9d2b4a828a54b3a5f7b3358b, State: Initialized, Role: FOLLOWER
I20260926 13:04:18.758101 9839 consensus_queue.cc:260] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.758230 9839 ts_tablet_manager.cc:1428] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:18.758523 9842 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: Bootstrap starting.
I20260926 13:04:18.758867 9842 tablet_bootstrap.cc:654] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:18.759249 9838 tablet_bootstrap.cc:654] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:18.759436 9842 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: No bootstrap required, opened a new log
I20260926 13:04:18.759490 9842 ts_tablet_manager.cc:1397] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:18.759618 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.758448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:44957)
0926 13:04:18.759550 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.759587 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1079,"thread_start_us":76,"threads_started":1}
I20260926 13:04:18.759610 9842 raft_consensus.cc:348] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.759678 9842 raft_consensus.cc:374] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:18.759779 9842 raft_consensus.cc:729] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ecf03c9d5d4cd78c391b46c8126ebb, State: Initialized, Role: FOLLOWER
I20260926 13:04:18.759842 9838 tablet_bootstrap.cc:492] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: No bootstrap required, opened a new log
I20260926 13:04:18.759845 9842 consensus_queue.cc:260] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.759900 9838 ts_tablet_manager.cc:1397] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 13:04:18.760016 9842 ts_tablet_manager.cc:1428] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:18.760040 9838 raft_consensus.cc:348] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.760113 9838 raft_consensus.cc:374] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:18.760143 9838 raft_consensus.cc:729] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b304cd6207334e4e8d01d0f95a14cea3, State: Initialized, Role: FOLLOWER
I20260926 13:04:18.760196 9838 consensus_queue.cc:260] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.760354 9838 ts_tablet_manager.cc:1428] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:18.760432 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.760209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:55365)
0926 13:04:18.760344 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.760394 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260926 13:04:18.760519 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.759884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:54127)
0926 13:04:18.760456 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.760492 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":65,"threads_started":1}
I20260926 13:04:18.760825 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.760736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:44419)
0926 13:04:18.760773 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.760795 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 13:04:18.760994 9471 heartbeater.cc:503] Master 127.2.86.60:40093 was elected leader, sending a full tablet report...
I20260926 13:04:18.761190 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.760764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:49939)
0926 13:04:18.761077 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.761112 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":76,"threads_started":1}
I20260926 13:04:18.761554 9607 heartbeater.cc:503] Master 127.2.86.60:40093 was elected leader, sending a full tablet report...
I20260926 13:04:18.828557 9841 raft_consensus.cc:482] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:18.828653 9841 raft_consensus.cc:504] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.828888 9841 leader_election.cc:288] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749), c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:18.829140 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea0ed1e16694b0aa2f1a9221741f557" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" is_pre_election: true
I20260926 13:04:18.829124 9561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea0ed1e16694b0aa2f1a9221741f557" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b304cd6207334e4e8d01d0f95a14cea3" is_pre_election: true
I20260926 13:04:18.829234 9561 raft_consensus.cc:2454] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 0.
I20260926 13:04:18.829234 9425 raft_consensus.cc:2454] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 0.
I20260926 13:04:18.829384 9631 leader_election.cc:302] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763bb84a9d2b4a828a54b3a5f7b3358b, c8ecf03c9d5d4cd78c391b46c8126ebb; no voters:
I20260926 13:04:18.829538 9841 raft_consensus.cc:2781] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:18.829589 9841 raft_consensus.cc:482] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:18.829612 9841 raft_consensus.cc:3037] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:18.830265 9841 raft_consensus.cc:504] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.830394 9841 leader_election.cc:288] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 election: Requested vote from peers b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749), c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749)
I20260926 13:04:18.830564 9561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea0ed1e16694b0aa2f1a9221741f557" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b304cd6207334e4e8d01d0f95a14cea3"
I20260926 13:04:18.830567 9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea0ed1e16694b0aa2f1a9221741f557" candidate_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb"
I20260926 13:04:18.830652 9425 raft_consensus.cc:3037] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:18.830672 9561 raft_consensus.cc:3037] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:18.831280 9425 raft_consensus.cc:2454] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 1.
I20260926 13:04:18.831298 9561 raft_consensus.cc:2454] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 763bb84a9d2b4a828a54b3a5f7b3358b in term 1.
I20260926 13:04:18.831445 9631 leader_election.cc:302] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 763bb84a9d2b4a828a54b3a5f7b3358b, c8ecf03c9d5d4cd78c391b46c8126ebb; no voters:
I20260926 13:04:18.831538 9841 raft_consensus.cc:2781] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:18.831668 9841 raft_consensus.cc:686] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 LEADER]: Becoming Leader. State: Replica: 763bb84a9d2b4a828a54b3a5f7b3358b, State: Running, Role: LEADER
I20260926 13:04:18.831709 9841 consensus_queue.cc:237] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } }
I20260926 13:04:18.831768 9743 heartbeater.cc:503] Master 127.2.86.60:40093 was elected leader, sending a full tablet report...
I20260926 13:04:18.832226 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.832125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:33815)
0926 13:04:18.832168 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.832196 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 13:04:18.832299 9289 catalog_manager.cc:5495] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b reported cstate change: term changed from 0 to 1, leader changed from <none> to 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3). New cstate: current_term: 1 leader_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "763bb84a9d2b4a828a54b3a5f7b3358b" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 45625 } health_report { overall_health: HEALTHY } } }
I20260926 13:04:18.845387 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.843197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.2.86.3:45625)
0926 13:04:18.843234 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:18.843236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:18.843244 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:18.843287 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:18.843288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:18.843299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:18.843350 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:18.843497 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.843976 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.844702 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:18.844965 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:18.845040 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:18.845175 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:18.845206 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:18.845247 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:18.845299 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 13:04:18.845318 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:04:18.846192 9425 raft_consensus.cc:1261] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Refusing update from remote peer 763bb84a9d2b4a828a54b3a5f7b3358b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:18.846208 9561 raft_consensus.cc:1261] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Refusing update from remote peer 763bb84a9d2b4a828a54b3a5f7b3358b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:18.846387 9851 consensus_queue.cc:1035] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ecf03c9d5d4cd78c391b46c8126ebb" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:18.846509 9841 consensus_queue.cc:1035] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b304cd6207334e4e8d01d0f95a14cea3" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:18.847169 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.847055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:52053)
0926 13:04:18.847094 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.847136 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 13:04:18.847431 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.847338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:43111)
0926 13:04:18.847374 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.847403 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 13:04:18.848042 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.847949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:43705)
0926 13:04:18.847985 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.848014 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 13:04:18.848913 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.848808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.1:55253)
0926 13:04:18.848842 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.848886 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 13:04:18.849431 9290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:18.849506 9290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:18.850010 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.849918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.2:40321)
0926 13:04:18.849955 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.849977 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 13:04:18.850419 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:18.850338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33835 (local address 127.2.86.3:41127)
0926 13:04:18.850373 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.850393 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 13:04:18.850731 9290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 851d33f1a0f3460ab83c1efef939d10c: Sending DeleteTablet for 3 replicas of tablet 9ea0ed1e16694b0aa2f1a9221741f557
I20260926 13:04:18.850986 9541 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:18 UTC) from {username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:60326
I20260926 13:04:18.851065 9677 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:18 UTC) from {username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:43716
I20260926 13:04:18.851174 9405 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea0ed1e16694b0aa2f1a9221741f557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:18 UTC) from {username='oryx', principal='oryx/127.2.86.60@KRBTEST.COM'} at 127.0.0.1:42310
I20260926 13:04:18.851212 9863 tablet_replica.cc:330] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: stopping tablet replica
I20260926 13:04:18.851223 9864 tablet_replica.cc:330] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: stopping tablet replica
I20260926 13:04:18.851257 9863 raft_consensus.cc:2229] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:18.851272 9864 raft_consensus.cc:2229] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:18.851302 9863 raft_consensus.cc:2258] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:18.851326 9864 raft_consensus.cc:2258] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:18.851481 9865 tablet_replica.cc:330] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: stopping tablet replica
I20260926 13:04:18.851533 9865 raft_consensus.cc:2229] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:18.851573 9865 raft_consensus.cc:2258] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:18.851600 9864 ts_tablet_manager.cc:1905] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:18.851634 9863 ts_tablet_manager.cc:1905] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:18.851747 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9342
I20260926 13:04:18.851917 9865 ts_tablet_manager.cc:1905] T 9ea0ed1e16694b0aa2f1a9221741f557 P c8ecf03c9d5d4cd78c391b46c8126ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:18.853413 9864 ts_tablet_manager.cc:1918] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:18.853463 9864 log.cc:1196] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/9ea0ed1e16694b0aa2f1a9221741f557
I20260926 13:04:18.853675 9864 ts_tablet_manager.cc:1939] T 9ea0ed1e16694b0aa2f1a9221741f557 P 763bb84a9d2b4a828a54b3a5f7b3358b: Deleting consensus metadata
I20260926 13:04:18.853684 9863 ts_tablet_manager.cc:1918] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:18.853739 9863 log.cc:1196] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/9ea0ed1e16694b0aa2f1a9221741f557
I20260926 13:04:18.853994 9863 ts_tablet_manager.cc:1939] T 9ea0ed1e16694b0aa2f1a9221741f557 P b304cd6207334e4e8d01d0f95a14cea3: Deleting consensus metadata
I20260926 13:04:18.854224 9276 catalog_manager.cc:4841] TS 763bb84a9d2b4a828a54b3a5f7b3358b (127.2.86.3:45625): tablet 9ea0ed1e16694b0aa2f1a9221741f557 (table test-table [id=ebc8e4d1cec64820bf50d4f09453b188]) successfully deleted
I20260926 13:04:18.857813 9274 catalog_manager.cc:4841] TS b304cd6207334e4e8d01d0f95a14cea3 (127.2.86.2:41749): tablet 9ea0ed1e16694b0aa2f1a9221741f557 (table test-table [id=ebc8e4d1cec64820bf50d4f09453b188]) successfully deleted
W20260926 13:04:18.858000 9835 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:18.857177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41749 (local address 127.0.0.1:42326)
0926 13:04:18.857216 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:18.857226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:18.857262 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:18.857946 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41749: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":22}
W20260926 13:04:18.858126 9275 catalog_manager.cc:4584] TS c8ecf03c9d5d4cd78c391b46c8126ebb (127.2.86.1:41749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ea0ed1e16694b0aa2f1a9221741f557: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41749: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:04:18.858247 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9478
I20260926 13:04:18.863408 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9614
I20260926 13:04:18.868742 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 8970
I20260926 13:04:18.873433 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9158
2026-09-26T13:04:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4554 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:18 dist-test-slave-j4lq krb5kdc[9868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:18 dist-test-slave-j4lq krb5kdc[9868](info): set up 2 sockets
Sep 26 13:04:18 dist-test-slave-j4lq krb5kdc[9868](info): commencing operation
krb5kdc: starting...
W20260926 13:04:19.917372 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:19 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab.
Entry for principal oryx/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab.
I20260926 13:04:20.033643 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40095
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab
--principal=oryx/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:20.112722 9881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:20.113027 9881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:20.113066 9881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:20.114269 9881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:20.114344 9881 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 13:04:20.114384 9881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:20.114421 9881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:20.114452 9881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:20.114491 9881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:20.116009 9881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40095
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.62.keytab
--principal=oryx/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:20.116278 9881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:20.116509 9881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:20.121192 9886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.121208 9889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:20.121472 9881 server_base.cc:1034] running on GCE node
W20260926 13:04:20.121218 9887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:20.121767 9881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:20.122087 9881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:20.123265 9881 hybrid_clock.cc:648] HybridClock initialized: now 1790427860123258 us; error 38 us; skew 500 ppm
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:20.125686 9881 init.cc:377] Logged in from keytab as oryx/127.2.86.62@KRBTEST.COM (short username oryx)
I20260926 13:04:20.126158 9881 webserver.cc:460] Webserver started at http://127.2.86.62:33943/ using document root <none> and password file <none>
I20260926 13:04:20.126387 9881 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:20.126466 9881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:20.126623 9881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:20.127674 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "49d5bee5972f4d41abdfb63873e5b9fc"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "603f8b69ae5ef214a4688e29d528f7d0"
server_key_iv: "6e8f0f71963e4fafa23408e8b901811e"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.128041 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "49d5bee5972f4d41abdfb63873e5b9fc"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "603f8b69ae5ef214a4688e29d528f7d0"
server_key_iv: "6e8f0f71963e4fafa23408e8b901811e"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.129441 9881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:20.130146 9896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.130306 9881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:20.130421 9881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "49d5bee5972f4d41abdfb63873e5b9fc"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "603f8b69ae5ef214a4688e29d528f7d0"
server_key_iv: "6e8f0f71963e4fafa23408e8b901811e"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.130496 9881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:20.139405 9881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:20.141197 9881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:20.141364 9881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:20.145972 9881 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40095
I20260926 13:04:20.146020 9958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40095 every 8 connection(s)
I20260926 13:04:20.146334 9881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:20.146917 9959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.148183 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9881
I20260926 13:04:20.148303 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:20.148524 2392 external_mini_cluster.cc:1428] Setting key 4a15a1438474d83e8e42a403ff02ddfa
I20260926 13:04:20.149632 9959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Bootstrap starting.
I20260926 13:04:20.150206 9959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.150480 9959 log.cc:824] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:20.151312 9959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: No bootstrap required, opened a new log
I20260926 13:04:20.153211 9959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } }
I20260926 13:04:20.153378 9959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.153414 9959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d5bee5972f4d41abdfb63873e5b9fc, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.153550 9959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } }
I20260926 13:04:20.153635 9959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:20.153681 9959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:20.153745 9959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:20.154284 9959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } }
I20260926 13:04:20.154366 9959 leader_election.cc:302] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49d5bee5972f4d41abdfb63873e5b9fc; no voters:
I20260926 13:04:20.154539 9959 leader_election.cc:288] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:20.154578 9964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:20.154695 9964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [term 1 LEADER]: Becoming Leader. State: Replica: 49d5bee5972f4d41abdfb63873e5b9fc, State: Running, Role: LEADER
I20260926 13:04:20.154806 9959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:20.154789 9964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } }
I20260926 13:04:20.155085 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d5bee5972f4d41abdfb63873e5b9fc. Latest consensus state: current_term: 1 leader_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } } }
I20260926 13:04:20.155144 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [sys.catalog]: This master's current role is: LEADER
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:20.155972 9968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:20.156514 9968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:20.156806 9966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d5bee5972f4d41abdfb63873e5b9fc" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40095 } } }
I20260926 13:04:20.156900 9966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:20.158212 9968 catalog_manager.cc:1347] Generated new cluster ID: 8c249b1c52024d2eb68740b6d46ee0df
I20260926 13:04:20.158260 9968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:20.159585 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.149062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.2.86.62:40095)
0926 13:04:20.149226 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.149236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.149267 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:04:20.149656 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.149662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.149705 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.149888 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.150566 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.151758 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.152526 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.153007 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.153123 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.155920 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.155923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.155924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.155935 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.158112 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.158518 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.158521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.158521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.158613 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.158788 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.158789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.158789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.159075 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.159189 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.159273 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.159398 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260926 13:04:20.165475 9968 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:20.166193 9968 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:20.175441 9968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Generated new TSK 0
I20260926 13:04:20.175642 9968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
I20260926 13:04:20.207852 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40095
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:20.287446 9985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:20.287650 9985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:20.287679 9985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:20.288853 9985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:20.288928 9985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:20.288949 9985 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:04:20.288996 9985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:20.290473 9985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:20.290715 9985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:20.290942 9985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:20.295881 9993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.295881 9991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.295923 9990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:20.296305 9985 server_base.cc:1034] running on GCE node
I20260926 13:04:20.296504 9985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:20.296734 9985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:20.297904 9985 hybrid_clock.cc:648] HybridClock initialized: now 1790427860297887 us; error 27 us; skew 500 ppm
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:20.300242 9985 init.cc:377] Logged in from keytab as oryx/127.2.86.1@KRBTEST.COM (short username oryx)
I20260926 13:04:20.300606 9985 webserver.cc:460] Webserver started at http://127.2.86.1:42657/ using document root <none> and password file <none>
I20260926 13:04:20.300776 9985 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:20.300815 9985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:20.300971 9985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:20.301962 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "1f5124a56a2983ef91ab7e76a7094016"
server_key_iv: "3da1d0cc9dc0570c523c2e325b881c81"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.302253 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "1f5124a56a2983ef91ab7e76a7094016"
server_key_iv: "3da1d0cc9dc0570c523c2e325b881c81"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.303432 9985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:20.304064 10000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.304203 9985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:20.304283 9985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "1f5124a56a2983ef91ab7e76a7094016"
server_key_iv: "3da1d0cc9dc0570c523c2e325b881c81"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.304332 9985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:20.315526 9985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:20.317214 9985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:20.317319 9985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:20.317806 9985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:20.317854 9985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.317883 9985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:20.317914 9985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.324587 9985 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:42573
I20260926 13:04:20.324645 10115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:42573 every 8 connection(s)
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:20.325016 9985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:20.327648 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.318194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41267 (local address 127.2.86.62:40095)
0926 13:04:20.318254 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.318256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.318438 (+ 182us) server_negotiation.cc:407] Connection header received
0926 13:04:20.318903 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.318904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.318917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.318998 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.319709 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.320554 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.322167 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.322545 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.322632 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.325438 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.325440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.325440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.325450 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.326591 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.326933 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.326933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.326934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.326990 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.327160 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.327160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.327161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.327256 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.327304 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.327525 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.327588 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:04:20.327790 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.318179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.1:41267)
0926 13:04:20.318313 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.318351 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.318815 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.319046 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.319049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.319314 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.319607 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.319613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.320666 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.320667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.322054 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.322056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.322649 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.322650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.322765 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.323373 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.323384 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.325333 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.326655 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.326661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.326663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.326864 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.327030 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.327030 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.327030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.327109 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.327310 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.327315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.327421 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.327501 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.327609 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260926 13:04:20.328280 10116 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40095
I20260926 13:04:20.328356 10116 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:20.328536 10116 heartbeater.cc:511] Master 127.2.86.62:40095 requested a full tablet report, sending...
I20260926 13:04:20.329123 9912 ts_manager.cc:194] Registered new tserver with Master: bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573)
I20260926 13:04:20.329968 9912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:41267
I20260926 13:04:20.332428 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 9985
I20260926 13:04:20.332562 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:20.332741 2392 external_mini_cluster.cc:1428] Setting key 357b0e8f4003a9c5bb81545c8d236a3c
I20260926 13:04:20.334584 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.329319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.1:41255)
0926 13:04:20.329428 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.329438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.329496 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.329622 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.329623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.329760 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.329808 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.329810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.330600 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.330601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.331109 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.331111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.331411 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.331412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.331456 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.331800 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.331804 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.332052 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.333762 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.333763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.333763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.333918 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.334111 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.334111 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.334112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.334153 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.334364 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.334365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.334400 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.334446 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.334517 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260926 13:04:20.334678 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.329379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41255 (local address 127.2.86.62:40095)
0926 13:04:20.329423 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.329425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.329480 (+ 55us) server_negotiation.cc:407] Connection header received
0926 13:04:20.329547 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.329548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.329557 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.329616 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.329867 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.330511 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.331176 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.331347 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.331407 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.332121 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.332122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.332122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.332127 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.333689 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.333988 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.333989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.333990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.334049 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.334188 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.334188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.334189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.334314 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.334366 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.334471 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.334545 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
WARNING: no policy specified for oryx/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
I20260926 13:04:20.355741 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40095
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:20.436437 10125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:20.436662 10125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:20.436702 10125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:20.437919 10125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:20.437985 10125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:20.438023 10125 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:04:20.438061 10125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:20.439510 10125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:20.439774 10125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:20.439988 10125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:20.444774 10133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.444779 10131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.444770 10130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:20.445112 10125 server_base.cc:1034] running on GCE node
I20260926 13:04:20.445300 10125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:20.445533 10125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:20.446668 10125 hybrid_clock.cc:648] HybridClock initialized: now 1790427860446660 us; error 30 us; skew 500 ppm
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:20.449039 10125 init.cc:377] Logged in from keytab as oryx/127.2.86.2@KRBTEST.COM (short username oryx)
I20260926 13:04:20.449400 10125 webserver.cc:460] Webserver started at http://127.2.86.2:45347/ using document root <none> and password file <none>
I20260926 13:04:20.449640 10125 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:20.449700 10125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:20.449786 10125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:20.450942 10125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "38b3b51f62314e72a139f9c03e206a9c"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "637fb138b262ecd0dba98c18a5e0c792"
server_key_iv: "8448c394e67625a0ad29de30095ac983"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.451366 10125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "38b3b51f62314e72a139f9c03e206a9c"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "637fb138b262ecd0dba98c18a5e0c792"
server_key_iv: "8448c394e67625a0ad29de30095ac983"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.452576 10125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:20.453245 10140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.453425 10125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.453518 10125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "38b3b51f62314e72a139f9c03e206a9c"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "637fb138b262ecd0dba98c18a5e0c792"
server_key_iv: "8448c394e67625a0ad29de30095ac983"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.453603 10125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:20.473716 10125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:20.475559 10125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:20.475720 10125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:20.476212 10125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:20.476274 10125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.476316 10125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:20.476344 10125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.482875 10125 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:46589
I20260926 13:04:20.482942 10255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:46589 every 8 connection(s)
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:20.483279 10125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:20.486121 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.476736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35277 (local address 127.2.86.62:40095)
0926 13:04:20.476830 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.476832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.477059 (+ 227us) server_negotiation.cc:407] Connection header received
0926 13:04:20.477582 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.477583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.477595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.477647 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.478423 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.479271 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.480809 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.481082 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.481172 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.483763 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.483765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.483765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.483772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.484846 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.485251 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.485251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.485252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.485299 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.485494 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.485495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.485495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.485585 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.485633 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.486004 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.486061 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 13:04:20.486269 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.476720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.2:35277)
0926 13:04:20.476926 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.476973 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.477473 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.477713 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.477717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.478036 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.478301 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.478308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.479377 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.479378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.480721 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.480724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.481182 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.481183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.481311 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.481788 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.481797 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.483678 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.484978 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.484988 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.484989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.485188 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.485356 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.485356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.485357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.485444 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.485748 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.485753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.485875 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.485969 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.486079 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":121,"threads_started":1}
I20260926 13:04:20.486703 10256 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40095
I20260926 13:04:20.486783 10256 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:20.486958 10256 heartbeater.cc:511] Master 127.2.86.62:40095 requested a full tablet report, sending...
I20260926 13:04:20.487358 9912 ts_manager.cc:194] Registered new tserver with Master: 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:20.487875 9912 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:35277
I20260926 13:04:20.490181 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10125
I20260926 13:04:20.490295 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:20.490497 2392 external_mini_cluster.cc:1428] Setting key 49559b129848c6faf183a6328fcaedb8
I20260926 13:04:20.493391 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.487790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:52583 (local address 127.2.86.62:40095)
0926 13:04:20.487828 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.487830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.487885 (+ 55us) server_negotiation.cc:407] Connection header received
0926 13:04:20.488000 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.488001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.488010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.488064 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.488439 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.489105 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.489775 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.489942 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.490008 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.490929 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.490930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.490931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.490937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.492250 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.492612 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.492613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.492613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.492666 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.492855 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.492856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.492857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.493023 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.493098 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.493242 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.493308 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:04:20.493407 10262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.487701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.2:52583)
0926 13:04:20.487825 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.487836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.487894 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.488103 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.488104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.488296 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.488370 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.488372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.489184 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.489185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.489712 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.489714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.490033 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.490034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.490103 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.490506 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.490511 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.490859 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.492365 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.492366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.492366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.492528 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.492730 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.492731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.492731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.492786 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.493091 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.493092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.493134 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.493218 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.493308 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
I20260926 13:04:20.513334 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40095
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:20.591154 10265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:20.591329 10265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:20.591351 10265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:20.592425 10265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:20.592463 10265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:20.592482 10265 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 13:04:20.592537 10265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:20.593997 10265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.86.62:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:20.594235 10265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:20.594447 10265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:20.599217 10271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.599226 10270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:20.599248 10273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:20.599416 10265 server_base.cc:1034] running on GCE node
I20260926 13:04:20.599617 10265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:20.599838 10265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:20.601001 10265 hybrid_clock.cc:648] HybridClock initialized: now 1790427860600988 us; error 28 us; skew 500 ppm
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:20.603289 10265 init.cc:377] Logged in from keytab as oryx/127.2.86.3@KRBTEST.COM (short username oryx)
I20260926 13:04:20.603703 10265 webserver.cc:460] Webserver started at http://127.2.86.3:45573/ using document root <none> and password file <none>
I20260926 13:04:20.603935 10265 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:20.604001 10265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:20.604131 10265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:20.605196 10265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "7fdb7effcb9c4065adc4561531ab5946"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "96e071da9dd07450b2700aa896fdc252"
server_key_iv: "0ef862a1f783c714f889136045396baa"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.605538 10265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "7fdb7effcb9c4065adc4561531ab5946"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "96e071da9dd07450b2700aa896fdc252"
server_key_iv: "0ef862a1f783c714f889136045396baa"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.606832 10265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:20.607626 10280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.607795 10265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:20.607887 10265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "7fdb7effcb9c4065adc4561531ab5946"
format_stamp: "Formatted at 2026-09-26 13:04:20 on dist-test-slave-j4lq"
server_key: "96e071da9dd07450b2700aa896fdc252"
server_key_iv: "0ef862a1f783c714f889136045396baa"
server_key_version: "encryptionkey@0"
I20260926 13:04:20.607975 10265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:20.616652 10265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:20.618854 10265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:20.619005 10265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:20.619445 10265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:20.619519 10265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.619563 10265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:20.619596 10265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:20.626513 10265 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:37447
I20260926 13:04:20.626600 10395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:37447 every 8 connection(s)
I20260926 13:04:20.626889 10265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:20.627177 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10265
I20260926 13:04:20.627272 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:20.627492 2392 external_mini_cluster.cc:1428] Setting key bcca5bf0b7fa5e7a985a2082bcd7e878
I20260926 13:04:20.629350 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.619971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:47249 (local address 127.2.86.62:40095)
0926 13:04:20.620009 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.620011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.620192 (+ 181us) server_negotiation.cc:407] Connection header received
0926 13:04:20.620672 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.620673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.620686 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.620740 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.621670 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.622539 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.623974 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.624181 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.624264 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.626331 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.626332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.626332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.626341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.627977 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.628443 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.628445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.628445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.628503 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.628748 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.628749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.628749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.628846 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.628966 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.629231 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.629291 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:04:20.629498 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.619946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.3:47249)
0926 13:04:20.620084 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.620114 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.620566 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.620847 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.620851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.621252 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.621564 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.621571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.622649 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.622650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.623881 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.623884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.624293 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.624294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.624414 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.624837 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.624844 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.626204 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.628050 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.628055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.628057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.628342 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.628569 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.628570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.628571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.628675 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.628957 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.628962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.629112 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.629202 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.629312 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260926 13:04:20.629933 10396 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40095
I20260926 13:04:20.630007 10396 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:20.630260 10396 heartbeater.cc:511] Master 127.2.86.62:40095 requested a full tablet report, sending...
I20260926 13:04:20.630645 9913 ts_manager.cc:194] Registered new tserver with Master: 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447)
I20260926 13:04:20.631161 9913 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:47249
I20260926 13:04:20.635816 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.630968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:41031 (local address 127.2.86.62:40095)
0926 13:04:20.631003 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.631004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.631061 (+ 57us) server_negotiation.cc:407] Connection header received
0926 13:04:20.631117 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.631118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.631125 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.631162 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.631527 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.631973 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.632657 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.632856 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.632973 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.633729 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.633731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.633731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.633737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.634800 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.635110 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.635111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.635112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.635159 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.635357 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.635358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.635358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.635460 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.635529 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.635671 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.635735 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:04:20.635833 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.630881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.2.86.3:41031)
0926 13:04:20.631004 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.631014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.631068 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.631176 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.631176 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.631382 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:20.631456 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.631458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.632064 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.632065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.632578 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.632580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.632992 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.632994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.633061 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.633383 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:20.633388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:20.633639 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:20.634880 (+ 1241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.634880 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.634881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.635041 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.635236 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:20.635237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:20.635237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:20.635289 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:20.635522 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:20.635523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:20.635562 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.635647 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.635736 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260926 13:04:20.638935 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:04:20.659735 9962 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:20.653732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.2.86.62:40095)
0926 13:04:20.653787 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.653789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.653798 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:20.653876 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.653877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.653889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.653958 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.654536 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.655092 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.656343 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.656562 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.656621 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.659626 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59904: BlockingRecv error: recv got EOF from 127.0.0.1:59904 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 26 13:04:20 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:20.666883 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.660502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.2.86.62:40095)
0926 13:04:20.660563 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.660565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.660646 (+ 81us) server_negotiation.cc:407] Connection header received
0926 13:04:20.660726 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.660727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.660737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.660792 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:20.661312 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.661743 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.662455 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.662633 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.662712 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.664578 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:20.664580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:20.664580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:20.664587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:20.665781 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.666145 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.666146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.666146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.666202 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:20.666401 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:20.666402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:20.666402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:20.666508 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:20.666608 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.666716 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.666785 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:04:20.668277 9913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:20.669909 9913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:20.676154 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46589 (local address 127.0.0.1:36724)
0926 13:04:20.672157 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.672219 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.672287 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.672520 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.672522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.672524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:20.672583 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.672584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.673731 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.673732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.674867 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.674869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.675848 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.675849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.675926 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.675939 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.676007 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.676058 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":46,"threads_started":1}
I20260926 13:04:20.676414 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.2.86.2:46589)
0926 13:04:20.672344 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.672350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.672360 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:20.672397 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.672399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.672413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.672504 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:20.672650 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.673657 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.674962 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.675682 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.675746 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.675812 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.676265 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.676303 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":71,"threads_started":1}
I20260926 13:04:20.676479 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.2.86.1:42573)
0926 13:04:20.672640 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.672645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.673125 (+ 480us) server_negotiation.cc:407] Connection header received
0926 13:04:20.673166 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.673169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.673183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.673257 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:20.673384 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.674354 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.675393 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.675900 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.675965 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.676039 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.676329 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.676369 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260926 13:04:20.676707 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.0.0.1:33286)
0926 13:04:20.672266 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.672277 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.672329 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.672903 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.672904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.672906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:20.672954 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.672955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.674206 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.674207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.675258 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.675260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.676136 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.676137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.676196 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.676294 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.676344 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.676377 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":73,"threads_started":1}
I20260926 13:04:20.676813 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42573 (local address 127.0.0.1:42470)
0926 13:04:20.673007 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.673017 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.673064 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.673273 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.673274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.673277 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:20.673323 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.673324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.674436 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.674438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.675310 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.675312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.676067 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.676068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.676140 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.676144 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.676194 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.676724 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":44,"threads_started":1}
I20260926 13:04:20.677066 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.672305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.2.86.3:37447)
0926 13:04:20.672686 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.672692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.672724 (+ 32us) server_negotiation.cc:407] Connection header received
0926 13:04:20.672762 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.672765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.672780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.672922 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:20.673043 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.674117 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.675350 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.676063 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.676128 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.676197 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.676930 (+ 733us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.676977 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":75,"threads_started":1}
I20260926 13:04:20.677207 10190 tablet_service.cc:1467] Processing CreateTablet for tablet 04a8184233094dc3aab41c002f42bda2 (DEFAULT_TABLE table=test-table [id=7a554f8d611d458d9510f2f431d6d990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.677548 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a8184233094dc3aab41c002f42bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.677462 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 04a8184233094dc3aab41c002f42bda2 (DEFAULT_TABLE table=test-table [id=7a554f8d611d458d9510f2f431d6d990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.677639 10330 tablet_service.cc:1467] Processing CreateTablet for tablet 04a8184233094dc3aab41c002f42bda2 (DEFAULT_TABLE table=test-table [id=7a554f8d611d458d9510f2f431d6d990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.677783 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a8184233094dc3aab41c002f42bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.677881 10330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a8184233094dc3aab41c002f42bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.680701 10421 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Bootstrap starting.
I20260926 13:04:20.681289 10421 tablet_bootstrap.cc:654] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.681551 10422 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Bootstrap starting.
I20260926 13:04:20.681653 10421 log.cc:824] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:20.681991 10423 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Bootstrap starting.
I20260926 13:04:20.682152 10422 tablet_bootstrap.cc:654] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.682446 10421 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: No bootstrap required, opened a new log
I20260926 13:04:20.682482 10422 log.cc:824] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:20.682529 10421 ts_tablet_manager.cc:1397] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:20.682580 10423 tablet_bootstrap.cc:654] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.682868 10423 log.cc:824] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:20.683228 10422 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: No bootstrap required, opened a new log
I20260926 13:04:20.683300 10422 ts_tablet_manager.cc:1397] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:20.683748 10423 tablet_bootstrap.cc:492] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: No bootstrap required, opened a new log
I20260926 13:04:20.683825 10423 ts_tablet_manager.cc:1397] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:20.684573 10421 raft_consensus.cc:348] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.684746 10421 raft_consensus.cc:374] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.684780 10421 raft_consensus.cc:729] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.684937 10421 consensus_queue.cc:260] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.685202 10421 ts_tablet_manager.cc:1428] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 13:04:20.685322 10396 heartbeater.cc:503] Master 127.2.86.62:40095 was elected leader, sending a full tablet report...
I20260926 13:04:20.685464 10423 raft_consensus.cc:348] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.685601 10423 raft_consensus.cc:374] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.685254 10422 raft_consensus.cc:348] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.685636 10423 raft_consensus.cc:729] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.685652 10422 raft_consensus.cc:374] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.685696 10422 raft_consensus.cc:729] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.685755 10423 consensus_queue.cc:260] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.685828 10422 consensus_queue.cc:260] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.685995 10423 ts_tablet_manager.cc:1428] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:20.686051 10256 heartbeater.cc:503] Master 127.2.86.62:40095 was elected leader, sending a full tablet report...
I20260926 13:04:20.686080 10422 ts_tablet_manager.cc:1428] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:04:20.686205 10116 heartbeater.cc:503] Master 127.2.86.62:40095 was elected leader, sending a full tablet report...
W20260926 13:04:20.733839 10257 tablet.cc:2369] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:04:20.825831 10117 tablet.cc:2369] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:04:20.875976 10429 raft_consensus.cc:482] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:20.876102 10429 raft_consensus.cc:504] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.876439 10429 leader_election.cc:288] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589), 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447)
W20260926 13:04:20.877586 10397 tablet.cc:2369] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:04:20.880506 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.876731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:51683 (local address 127.2.86.2:46589)
0926 13:04:20.876797 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.876799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.876892 (+ 93us) server_negotiation.cc:407] Connection header received
0926 13:04:20.877013 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.877014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.877025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.877090 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:20.877270 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.878022 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.879600 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.880143 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.880217 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.880228 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.880398 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.880434 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:20.880502 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.876789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:34845 (local address 127.2.86.3:37447)
0926 13:04:20.876833 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.876835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.876891 (+ 56us) server_negotiation.cc:407] Connection header received
0926 13:04:20.877013 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.877014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.877025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.877090 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:20.877274 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.878023 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.879599 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.880143 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.880217 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.880228 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.880399 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.880433 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:04:20.880551 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.876642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46589 (local address 127.2.86.1:51683)
0926 13:04:20.876808 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.876822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.876944 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.877107 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.877107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.877111 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:20.877195 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.877198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.878111 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.878111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.879507 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.879510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.880235 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.880235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.880313 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.880321 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.880381 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.880432 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":99,"thread_start_us":77,"threads_started":1}
I20260926 13:04:20.880756 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.876675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.2.86.1:34845)
0926 13:04:20.876815 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:20.876824 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:20.876917 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:20.877107 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:20.877107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:20.877110 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:20.877195 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.877197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.878111 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.878111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.879507 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:20.879510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.880233 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.880234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:20.880313 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.880320 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:04:20.880381 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 13:04:20.880432 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":104,"thread_start_us":61,"threads_started":1}
I20260926 13:04:20.880949 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a8184233094dc3aab41c002f42bda2" candidate_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c" is_pre_election: true
I20260926 13:04:20.881114 10210 raft_consensus.cc:2454] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfb3f23fdd2c4dd5826fc610ff55ee51 in term 0.
I20260926 13:04:20.881153 10350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a8184233094dc3aab41c002f42bda2" candidate_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb7effcb9c4065adc4561531ab5946" is_pre_election: true
I20260926 13:04:20.881276 10350 raft_consensus.cc:2454] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfb3f23fdd2c4dd5826fc610ff55ee51 in term 0.
I20260926 13:04:20.881361 10004 leader_election.cc:302] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b51f62314e72a139f9c03e206a9c, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:20.881520 10429 raft_consensus.cc:2781] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:20.881613 10429 raft_consensus.cc:482] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:20.881654 10429 raft_consensus.cc:3037] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:20.882321 10429 raft_consensus.cc:504] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.882471 10429 leader_election.cc:288] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589), 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447)
I20260926 13:04:20.882599 10350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a8184233094dc3aab41c002f42bda2" candidate_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb7effcb9c4065adc4561531ab5946"
I20260926 13:04:20.882619 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a8184233094dc3aab41c002f42bda2" candidate_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c"
I20260926 13:04:20.882669 10350 raft_consensus.cc:3037] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:20.882728 10210 raft_consensus.cc:3037] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:20.883349 10350 raft_consensus.cc:2454] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfb3f23fdd2c4dd5826fc610ff55ee51 in term 1.
I20260926 13:04:20.883517 10210 raft_consensus.cc:2454] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfb3f23fdd2c4dd5826fc610ff55ee51 in term 1.
I20260926 13:04:20.883523 10002 leader_election.cc:302] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdb7effcb9c4065adc4561531ab5946, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:20.883731 10429 raft_consensus.cc:2781] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:20.883867 10429 raft_consensus.cc:686] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 LEADER]: Becoming Leader. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Running, Role: LEADER
I20260926 13:04:20.883960 10429 consensus_queue.cc:237] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.884694 9912 catalog_manager.cc:5495] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:20.901042 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:20.898633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.2.86.1:42573)
0926 13:04:20.898673 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:20.898675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:20.898684 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:20.898760 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:20.898761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:20.898780 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:20.898865 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:20.899026 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.899488 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.900263 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:20.900468 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:20.900535 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:20.900706 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:20.900808 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:20.900925 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:20.900945 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 13:04:20.900963 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:20.902717 10210 raft_consensus.cc:1261] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Refusing update from remote peer bfb3f23fdd2c4dd5826fc610ff55ee51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:20.902740 10350 raft_consensus.cc:1261] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Refusing update from remote peer bfb3f23fdd2c4dd5826fc610ff55ee51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:20.903039 10429 consensus_queue.cc:1035] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:20.903178 10432 consensus_queue.cc:1035] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:20.905571 10436 mvcc.cc:205] Tried to move back new op lower bound from 7333592518254968832 to 7333592518181257216. Current Snapshot: MvccSnapshot[applied={T|T < 7333592518254968832}]
I20260926 13:04:20.906139 10435 mvcc.cc:205] Tried to move back new op lower bound from 7333592518254968832 to 7333592518181257216. Current Snapshot: MvccSnapshot[applied={T|T < 7333592518254968832 or (T in {7333592518254968832})}]
I20260926 13:04:20.906297 10437 mvcc.cc:205] Tried to move back new op lower bound from 7333592518254968832 to 7333592518181257216. Current Snapshot: MvccSnapshot[applied={T|T < 7333592518254968832}]
I20260926 13:04:20.907816 9912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:20.907925 9912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:20.908658 9912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Sending DeleteTablet for 3 replicas of tablet 04a8184233094dc3aab41c002f42bda2
I20260926 13:04:20.908969 10190 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a8184233094dc3aab41c002f42bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:20 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36724
I20260926 13:04:20.909070 10330 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a8184233094dc3aab41c002f42bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:20 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33286
I20260926 13:04:20.909163 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a8184233094dc3aab41c002f42bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:20 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42470
I20260926 13:04:20.909232 9912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:20.909371 9912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:20.909561 10444 tablet_replica.cc:330] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: stopping tablet replica
I20260926 13:04:20.909657 10444 raft_consensus.cc:2229] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:20.909768 10444 raft_consensus.cc:2258] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:20.910234 10445 tablet_replica.cc:330] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: stopping tablet replica
I20260926 13:04:20.910333 10445 raft_consensus.cc:2229] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:20.910441 10445 raft_consensus.cc:2258] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:20.910629 10190 tablet_service.cc:1467] Processing CreateTablet for tablet 95f201fe91774b0e9f233dcfb8750189 (DEFAULT_TABLE table=test-table [id=127d5cd388794598957b68ce8e7c6753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.910751 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f201fe91774b0e9f233dcfb8750189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.910888 10445 ts_tablet_manager.cc:1905] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:20.912117 10423 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Bootstrap starting.
I20260926 13:04:20.912289 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 95f201fe91774b0e9f233dcfb8750189 (DEFAULT_TABLE table=test-table [id=127d5cd388794598957b68ce8e7c6753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.912392 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f201fe91774b0e9f233dcfb8750189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.912547 10423 tablet_bootstrap.cc:654] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.912590 10445 ts_tablet_manager.cc:1918] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:20.912667 10445 log.cc:1196] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/04a8184233094dc3aab41c002f42bda2
I20260926 13:04:20.912968 10445 ts_tablet_manager.cc:1939] T 04a8184233094dc3aab41c002f42bda2 P 7fdb7effcb9c4065adc4561531ab5946: Deleting consensus metadata
I20260926 13:04:20.913671 10422 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Bootstrap starting.
I20260926 13:04:20.913718 9898 catalog_manager.cc:4841] TS 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447): tablet 04a8184233094dc3aab41c002f42bda2 (table test-table [id=7a554f8d611d458d9510f2f431d6d990]) successfully deleted
I20260926 13:04:20.913882 10446 tablet_replica.cc:330] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: stopping tablet replica
I20260926 13:04:20.913940 10330 tablet_service.cc:1467] Processing CreateTablet for tablet 95f201fe91774b0e9f233dcfb8750189 (DEFAULT_TABLE table=test-table [id=127d5cd388794598957b68ce8e7c6753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:20.913978 10446 raft_consensus.cc:2229] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:20.914029 10330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f201fe91774b0e9f233dcfb8750189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:20.914064 10446 raft_consensus.cc:2258] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:20.914089 10422 tablet_bootstrap.cc:654] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.914476 10423 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: No bootstrap required, opened a new log
I20260926 13:04:20.914481 10446 ts_tablet_manager.cc:1905] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:20.914520 10423 ts_tablet_manager.cc:1397] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:20.914666 10423 raft_consensus.cc:348] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.914739 10423 raft_consensus.cc:374] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.914762 10423 raft_consensus.cc:729] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.914803 10423 consensus_queue.cc:260] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.914815 10422 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: No bootstrap required, opened a new log
I20260926 13:04:20.914855 10422 ts_tablet_manager.cc:1397] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:20.914885 10423 ts_tablet_manager.cc:1428] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:20.914985 10422 raft_consensus.cc:348] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.915040 10422 raft_consensus.cc:374] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.915058 10422 raft_consensus.cc:729] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.915102 10422 consensus_queue.cc:260] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.915184 10422 ts_tablet_manager.cc:1428] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:20.915315 10421 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Bootstrap starting.
I20260926 13:04:20.915704 10421 tablet_bootstrap.cc:654] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:20.915957 10444 ts_tablet_manager.cc:1905] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:20.916018 10446 ts_tablet_manager.cc:1918] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:20.916090 10446 log.cc:1196] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/04a8184233094dc3aab41c002f42bda2
I20260926 13:04:20.916352 10446 ts_tablet_manager.cc:1939] T 04a8184233094dc3aab41c002f42bda2 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting consensus metadata
I20260926 13:04:20.916580 10421 tablet_bootstrap.cc:492] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: No bootstrap required, opened a new log
I20260926 13:04:20.916631 10421 ts_tablet_manager.cc:1397] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:04:20.916715 9900 catalog_manager.cc:4841] TS bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573): tablet 04a8184233094dc3aab41c002f42bda2 (table test-table [id=7a554f8d611d458d9510f2f431d6d990]) successfully deleted
I20260926 13:04:20.916810 10421 raft_consensus.cc:348] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.916898 10421 raft_consensus.cc:374] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:20.916919 10421 raft_consensus.cc:729] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Initialized, Role: FOLLOWER
I20260926 13:04:20.916963 10421 consensus_queue.cc:260] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:20.917049 10421 ts_tablet_manager.cc:1428] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:20.917567 10444 ts_tablet_manager.cc:1918] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:20.917645 10444 log.cc:1196] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/04a8184233094dc3aab41c002f42bda2
I20260926 13:04:20.917856 10444 ts_tablet_manager.cc:1939] T 04a8184233094dc3aab41c002f42bda2 P 38b3b51f62314e72a139f9c03e206a9c: Deleting consensus metadata
I20260926 13:04:20.918200 9900 catalog_manager.cc:4841] TS 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589): tablet 04a8184233094dc3aab41c002f42bda2 (table test-table [id=7a554f8d611d458d9510f2f431d6d990]) successfully deleted
I20260926 13:04:21.010514 10427 raft_consensus.cc:482] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:21.010603 10427 raft_consensus.cc:504] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.010947 10427 leader_election.cc:288] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.014408 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.011424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:52803 (local address 127.2.86.2:46589)
0926 13:04:21.011482 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.011484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.011536 (+ 52us) server_negotiation.cc:407] Connection header received
0926 13:04:21.011577 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.011578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.011590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.011657 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:21.011807 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.012485 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.013666 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.014012 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.014114 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.014124 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.014282 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.014324 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:04:21.014494 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.011127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42573 (local address 127.2.86.3:38963)
0926 13:04:21.011184 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.011197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.011254 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.011414 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.011415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.011417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:21.011489 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.011491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.012235 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.012236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.013307 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.013309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.014174 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.014175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.014258 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.014262 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.014312 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.014356 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 13:04:21.014562 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.011308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46589 (local address 127.2.86.3:52803)
0926 13:04:21.011466 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.011476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.011527 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.011661 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.011661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.011663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:21.011734 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.011735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.012571 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.012572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.013586 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.013588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.014141 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.014142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.014205 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.014209 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.014276 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.014316 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":57,"threads_started":1}
I20260926 13:04:21.014791 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.011230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38963 (local address 127.2.86.1:42573)
0926 13:04:21.011274 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.011276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.011284 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:21.011313 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.011313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.011322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.011371 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:21.011569 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.012146 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.013403 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.013994 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.014131 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.014143 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.014687 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.014731 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:04:21.014902 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f201fe91774b0e9f233dcfb8750189" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c" is_pre_election: true
I20260926 13:04:21.015004 10210 raft_consensus.cc:2454] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.015071 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f201fe91774b0e9f233dcfb8750189" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" is_pre_election: true
I20260926 13:04:21.015200 10070 raft_consensus.cc:2454] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.015177 10284 leader_election.cc:302] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b51f62314e72a139f9c03e206a9c, 7fdb7effcb9c4065adc4561531ab5946; no voters:
I20260926 13:04:21.015287 10427 raft_consensus.cc:2781] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:21.015323 10427 raft_consensus.cc:482] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:21.015344 10427 raft_consensus.cc:3037] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.016001 10427 raft_consensus.cc:504] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.016088 10427 leader_election.cc:288] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Requested vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.016258 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f201fe91774b0e9f233dcfb8750189" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
I20260926 13:04:21.016322 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f201fe91774b0e9f233dcfb8750189" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c"
I20260926 13:04:21.016355 10070 raft_consensus.cc:3037] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.016384 10210 raft_consensus.cc:3037] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.017141 10070 raft_consensus.cc:2454] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.017141 10210 raft_consensus.cc:2454] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.017302 10284 leader_election.cc:302] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdb7effcb9c4065adc4561531ab5946, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.017419 10427 raft_consensus.cc:2781] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:21.017558 10427 raft_consensus.cc:686] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 LEADER]: Becoming Leader. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Running, Role: LEADER
I20260926 13:04:21.017632 10427 consensus_queue.cc:237] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.018324 9912 catalog_manager.cc:5495] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } health_report { overall_health: HEALTHY } } }
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427860, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.62@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:21.056260 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.049486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.2.86.62:40095)
0926 13:04:21.049554 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.049556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.049652 (+ 96us) server_negotiation.cc:407] Connection header received
0926 13:04:21.049735 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.049736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.049746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.049821 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:21.050290 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.050758 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.051426 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.051604 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.051687 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.053760 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:21.053761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:21.053761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:21.053767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:21.055076 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:21.055417 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:21.055418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:21.055418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:21.055474 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:21.055657 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:21.055658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:21.055658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:21.055798 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:21.055874 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.056025 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.056114 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":40}
I20260926 13:04:21.056260 10458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.049402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40095 (local address 127.0.0.1:59920)
0926 13:04:21.049586 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.049595 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.049671 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.049826 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.049827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.050170 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:21.050211 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.050214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.050836 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.050838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.051340 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.051341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.051687 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.051688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.051734 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.052298 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:21.052303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:21.053680 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:21.055158 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:21.055164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:21.055166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:21.055337 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:21.055536 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:21.055537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:21.055538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:21.055592 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:21.055865 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:21.055871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:21.055912 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.056002 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.056164 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":66,"thread_start_us":96,"threads_started":1}
I20260926 13:04:21.058146 9912 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:59920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 13:04:21.058331 9912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:21.059711 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 36d4bc9a769a4e109a92a06a2435c38d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=390ab9e04af24e95abc8222948c07313]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:04:21.059732 10330 tablet_service.cc:1467] Processing CreateTablet for tablet 36d4bc9a769a4e109a92a06a2435c38d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=390ab9e04af24e95abc8222948c07313]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:04:21.059861 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36d4bc9a769a4e109a92a06a2435c38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.059882 10330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36d4bc9a769a4e109a92a06a2435c38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.061066 10422 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: Bootstrap starting.
I20260926 13:04:21.061138 10190 tablet_service.cc:1467] Processing CreateTablet for tablet 36d4bc9a769a4e109a92a06a2435c38d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=390ab9e04af24e95abc8222948c07313]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 13:04:21.061282 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36d4bc9a769a4e109a92a06a2435c38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.061285 10421 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: Bootstrap starting.
I20260926 13:04:21.061378 10422 tablet_bootstrap.cc:654] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.061676 10421 tablet_bootstrap.cc:654] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.061898 10422 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: No bootstrap required, opened a new log
I20260926 13:04:21.061964 10422 ts_tablet_manager.cc:1397] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:21.062090 10422 raft_consensus.cc:348] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.062153 10422 raft_consensus.cc:374] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.062167 10422 raft_consensus.cc:729] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.062224 10422 consensus_queue.cc:260] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.062320 10422 ts_tablet_manager.cc:1428] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.062331 10432 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.062427 10432 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.062788 10423 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: Bootstrap starting.
I20260926 13:04:21.063138 10423 tablet_bootstrap.cc:654] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.063369 10421 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: No bootstrap required, opened a new log
I20260926 13:04:21.063401 10421 ts_tablet_manager.cc:1397] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:21.063503 10421 raft_consensus.cc:348] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.063555 10421 raft_consensus.cc:374] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.063568 10421 raft_consensus.cc:729] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.063633 10421 consensus_queue.cc:260] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.063658 10423 tablet_bootstrap.cc:492] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: No bootstrap required, opened a new log
I20260926 13:04:21.063702 10423 ts_tablet_manager.cc:1397] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:21.063735 10421 ts_tablet_manager.cc:1428] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.063849 10423 raft_consensus.cc:348] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.063927 10423 raft_consensus.cc:374] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.063952 10423 raft_consensus.cc:729] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.063994 10423 consensus_queue.cc:260] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.064083 10423 ts_tablet_manager.cc:1428] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.064194 10427 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.064281 10427 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.064411 10428 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.064489 10428 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.160211 10427 raft_consensus.cc:482] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:21.160301 10427 raft_consensus.cc:504] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.160511 10427 leader_election.cc:288] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.160741 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36d4bc9a769a4e109a92a06a2435c38d" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" is_pre_election: true
I20260926 13:04:21.160756 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36d4bc9a769a4e109a92a06a2435c38d" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c" is_pre_election: true
I20260926 13:04:21.160845 10210 raft_consensus.cc:2454] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.160845 10070 raft_consensus.cc:2454] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.161017 10284 leader_election.cc:302] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdb7effcb9c4065adc4561531ab5946, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.161123 10427 raft_consensus.cc:2781] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:21.161166 10427 raft_consensus.cc:482] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:21.161197 10427 raft_consensus.cc:3037] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.161832 10427 raft_consensus.cc:504] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.161960 10427 leader_election.cc:288] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Requested vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.162127 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36d4bc9a769a4e109a92a06a2435c38d" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c"
I20260926 13:04:21.162142 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36d4bc9a769a4e109a92a06a2435c38d" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
I20260926 13:04:21.162205 10070 raft_consensus.cc:3037] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.162205 10210 raft_consensus.cc:3037] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.162938 10070 raft_consensus.cc:2454] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.162935 10210 raft_consensus.cc:2454] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.163103 10284 leader_election.cc:302] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdb7effcb9c4065adc4561531ab5946, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.163223 10427 raft_consensus.cc:2781] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:21.163290 10427 raft_consensus.cc:686] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [term 1 LEADER]: Becoming Leader. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Running, Role: LEADER
I20260926 13:04:21.163337 10427 consensus_queue.cc:237] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.163654 10451 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: TxnStatusTablet state changed. Reason: New leader 7fdb7effcb9c4065adc4561531ab5946. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.163745 10451 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:04:21.163885 10463 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 13:04:21.163939 9912 catalog_manager.cc:5495] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } health_report { overall_health: HEALTHY } } }
I20260926 13:04:21.164023 10463 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 13:04:21.179203 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.176377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.2.86.3:37447)
0926 13:04:21.176416 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.176419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.176429 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:21.176525 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.176526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.176540 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.176597 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.176729 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.177398 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.178481 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.178646 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.178693 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.178888 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.178978 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.179029 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.179120 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.179136 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:04:21.179241 10458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.176315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.0.0.1:33294)
0926 13:04:21.176392 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.176403 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.176443 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.176621 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.176622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.176623 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:04:21.176671 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.176674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.177472 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.177473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.178389 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.178391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.178711 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.178712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.178775 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.178789 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.179057 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.179064 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.179118 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.179165 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 13:04:21.180292 10210 raft_consensus.cc:1261] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.180373 10070 raft_consensus.cc:1261] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.180533 10427 consensus_queue.cc:1035] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.180678 10451 consensus_queue.cc:1035] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.181280 10428 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: TxnStatusTablet state changed. Reason: New leader 7fdb7effcb9c4065adc4561531ab5946. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.181295 10432 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: TxnStatusTablet state changed. Reason: New leader 7fdb7effcb9c4065adc4561531ab5946. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.181365 10428 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.181372 10432 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.182025 10451 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.182086 10427 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.182122 10451 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:04:21.182137 10427 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 7fdb7effcb9c4065adc4561531ab5946: This TxnStatusTablet replica's current role is: LEADER
I20260926 13:04:21.182588 10428 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.182658 10428 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P 38b3b51f62314e72a139f9c03e206a9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.183039 10432 tablet_replica.cc:439] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } } }
I20260926 13:04:21.183139 10432 tablet_replica.cc:442] T 36d4bc9a769a4e109a92a06a2435c38d P bfb3f23fdd2c4dd5826fc610ff55ee51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 13:04:21.186035 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.183848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.2.86.3:37447)
0926 13:04:21.183901 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.183903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.183911 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:21.183952 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.183953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.183961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.184014 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.184120 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.184553 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.185336 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.185531 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.185610 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.185763 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.185791 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.185877 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.185954 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.185969 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:04:21.190701 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.188272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:43287 (local address 127.2.86.3:37447)
0926 13:04:21.188308 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.188310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.188430 (+ 120us) server_negotiation.cc:407] Connection header received
0926 13:04:21.188490 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.188491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.188497 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.188537 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.188645 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.189091 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.190163 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.190307 (+ 144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.190347 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.190531 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.190551 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.190582 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.190634 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.190649 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260926 13:04:21.190797 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.188196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.2.86.3:43287)
0926 13:04:21.188351 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.188360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.188437 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.188543 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.188544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.188545 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:04:21.188593 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.188594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.189160 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.189161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.190080 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.190082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.190369 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.190370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.190428 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.190457 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.190586 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.190587 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.190633 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.190671 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":99,"threads_started":1}
I20260926 13:04:21.192667 10070 raft_consensus.cc:1261] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.192672 10210 raft_consensus.cc:1261] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.192852 10427 consensus_queue.cc:1035] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.193109 10468 consensus_queue.cc:1035] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.204128 9913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.204214 9913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.204973 9913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Sending DeleteTablet for 3 replicas of tablet 95f201fe91774b0e9f233dcfb8750189
I20260926 13:04:21.205196 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f201fe91774b0e9f233dcfb8750189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42470
I20260926 13:04:21.205238 10190 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f201fe91774b0e9f233dcfb8750189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36724
I20260926 13:04:21.205291 10446 tablet_replica.cc:330] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: stopping tablet replica
I20260926 13:04:21.205320 10444 tablet_replica.cc:330] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: stopping tablet replica
I20260926 13:04:21.205341 10446 raft_consensus.cc:2229] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.205361 10444 raft_consensus.cc:2229] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.205374 10446 raft_consensus.cc:2258] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.205390 10444 raft_consensus.cc:2258] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.205567 10329 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f201fe91774b0e9f233dcfb8750189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33286
I20260926 13:04:21.205660 10445 tablet_replica.cc:330] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: stopping tablet replica
I20260926 13:04:21.205672 10446 ts_tablet_manager.cc:1905] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.205719 10445 raft_consensus.cc:2229] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:21.205842 10445 raft_consensus.cc:2258] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.205853 10444 ts_tablet_manager.cc:1905] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.207203 10446 ts_tablet_manager.cc:1918] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:21.207283 10446 log.cc:1196] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/95f201fe91774b0e9f233dcfb8750189
I20260926 13:04:21.207463 10445 ts_tablet_manager.cc:1905] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.207535 10446 ts_tablet_manager.cc:1939] T 95f201fe91774b0e9f233dcfb8750189 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting consensus metadata
I20260926 13:04:21.207830 9900 catalog_manager.cc:4841] TS bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573): tablet 95f201fe91774b0e9f233dcfb8750189 (table test-table [id=127d5cd388794598957b68ce8e7c6753]) successfully deleted
I20260926 13:04:21.208056 10444 ts_tablet_manager.cc:1918] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:21.208109 10444 log.cc:1196] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/95f201fe91774b0e9f233dcfb8750189
I20260926 13:04:21.208328 10444 ts_tablet_manager.cc:1939] T 95f201fe91774b0e9f233dcfb8750189 P 38b3b51f62314e72a139f9c03e206a9c: Deleting consensus metadata
I20260926 13:04:21.208616 9900 catalog_manager.cc:4841] TS 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589): tablet 95f201fe91774b0e9f233dcfb8750189 (table test-table [id=127d5cd388794598957b68ce8e7c6753]) successfully deleted
I20260926 13:04:21.209167 10445 ts_tablet_manager.cc:1918] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:21.209224 10445 log.cc:1196] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/95f201fe91774b0e9f233dcfb8750189
I20260926 13:04:21.209420 10445 ts_tablet_manager.cc:1939] T 95f201fe91774b0e9f233dcfb8750189 P 7fdb7effcb9c4065adc4561531ab5946: Deleting consensus metadata
I20260926 13:04:21.209751 9898 catalog_manager.cc:4841] TS 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447): tablet 95f201fe91774b0e9f233dcfb8750189 (table test-table [id=127d5cd388794598957b68ce8e7c6753]) successfully deleted
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[9868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:04:21.230739 9962 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:21.224734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59924 (local address 127.2.86.62:40095)
0926 13:04:21.224794 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.224796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.224947 (+ 151us) server_negotiation.cc:407] Connection header received
0926 13:04:21.225023 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.225024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.225035 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.225098 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:21.225695 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.226605 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.227325 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.227631 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.227743 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.230654 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59924: BlockingRecv error: recv got EOF from 127.0.0.1:59924 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[9868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM
I20260926 13:04:21.238837 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.231476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.2.86.62:40095)
0926 13:04:21.231516 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.231518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.231571 (+ 53us) server_negotiation.cc:407] Connection header received
0926 13:04:21.231627 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.231627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.231636 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.231690 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:21.232085 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.232922 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.233584 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.233873 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.233948 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.236256 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:21.236257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:21.236257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:21.236265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:21.237821 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:21.238129 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:21.238130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:21.238130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:21.238191 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:21.238365 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:21.238366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:21.238367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:21.238497 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:21.238597 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.238674 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.238742 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:04:21.240236 9913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:21.240402 9913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:21.241844 10050 tablet_service.cc:1467] Processing CreateTablet for tablet e581eb16305446a3b444b4e543f2d2a6 (DEFAULT_TABLE table=test-table [id=35e15b18474f402d9fe626073e2a0b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.241897 10190 tablet_service.cc:1467] Processing CreateTablet for tablet e581eb16305446a3b444b4e543f2d2a6 (DEFAULT_TABLE table=test-table [id=35e15b18474f402d9fe626073e2a0b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.241892 10329 tablet_service.cc:1467] Processing CreateTablet for tablet e581eb16305446a3b444b4e543f2d2a6 (DEFAULT_TABLE table=test-table [id=35e15b18474f402d9fe626073e2a0b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.241981 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e581eb16305446a3b444b4e543f2d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.242014 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e581eb16305446a3b444b4e543f2d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.242014 10329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e581eb16305446a3b444b4e543f2d2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.243340 10422 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Bootstrap starting.
I20260926 13:04:21.243374 10421 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Bootstrap starting.
I20260926 13:04:21.243378 10423 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Bootstrap starting.
I20260926 13:04:21.243659 10421 tablet_bootstrap.cc:654] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.243675 10422 tablet_bootstrap.cc:654] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.243752 10423 tablet_bootstrap.cc:654] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.244447 10421 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: No bootstrap required, opened a new log
I20260926 13:04:21.244498 10421 ts_tablet_manager.cc:1397] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:21.244499 10423 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: No bootstrap required, opened a new log
I20260926 13:04:21.244545 10423 ts_tablet_manager.cc:1397] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:21.244630 10421 raft_consensus.cc:348] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.244673 10422 tablet_bootstrap.cc:492] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: No bootstrap required, opened a new log
I20260926 13:04:21.244694 10421 raft_consensus.cc:374] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.244715 10421 raft_consensus.cc:729] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.244714 10422 ts_tablet_manager.cc:1397] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:21.244700 10423 raft_consensus.cc:348] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.244768 10423 raft_consensus.cc:374] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.244789 10423 raft_consensus.cc:729] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.244763 10421 consensus_queue.cc:260] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.244810 10422 raft_consensus.cc:348] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.244829 10423 consensus_queue.cc:260] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.244899 10422 raft_consensus.cc:374] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.244922 10422 raft_consensus.cc:729] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.244968 10423 ts_tablet_manager.cc:1428] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:21.244961 10422 consensus_queue.cc:260] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.245051 10422 ts_tablet_manager.cc:1428] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.245826 10421 ts_tablet_manager.cc:1428] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:04:21.375579 10427 raft_consensus.cc:482] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:21.375697 10427 raft_consensus.cc:504] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.375878 10427 leader_election.cc:288] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.376150 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e581eb16305446a3b444b4e543f2d2a6" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" is_pre_election: true
I20260926 13:04:21.376147 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e581eb16305446a3b444b4e543f2d2a6" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c" is_pre_election: true
I20260926 13:04:21.376251 10210 raft_consensus.cc:2454] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.376255 10069 raft_consensus.cc:2454] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 0.
I20260926 13:04:21.376448 10284 leader_election.cc:302] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b51f62314e72a139f9c03e206a9c, 7fdb7effcb9c4065adc4561531ab5946; no voters:
I20260926 13:04:21.376583 10427 raft_consensus.cc:2781] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:21.376638 10427 raft_consensus.cc:482] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:21.376660 10427 raft_consensus.cc:3037] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.377609 10427 raft_consensus.cc:504] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.377737 10427 leader_election.cc:288] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Requested vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589)
I20260926 13:04:21.377907 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e581eb16305446a3b444b4e543f2d2a6" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
I20260926 13:04:21.377914 10210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e581eb16305446a3b444b4e543f2d2a6" candidate_uuid: "7fdb7effcb9c4065adc4561531ab5946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b51f62314e72a139f9c03e206a9c"
I20260926 13:04:21.378005 10210 raft_consensus.cc:3037] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.378005 10069 raft_consensus.cc:3037] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.378686 10069 raft_consensus.cc:2454] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.378787 10210 raft_consensus.cc:2454] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdb7effcb9c4065adc4561531ab5946 in term 1.
I20260926 13:04:21.378856 10284 leader_election.cc:302] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdb7effcb9c4065adc4561531ab5946, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.378955 10427 raft_consensus.cc:2781] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:21.379019 10427 raft_consensus.cc:686] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 LEADER]: Becoming Leader. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Running, Role: LEADER
I20260926 13:04:21.379068 10427 consensus_queue.cc:237] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.379637 9912 catalog_manager.cc:5495] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "7fdb7effcb9c4065adc4561531ab5946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } health_report { overall_health: HEALTHY } } }
I20260926 13:04:21.391620 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.389381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.2.86.3:37447)
0926 13:04:21.389434 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.389436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.389445 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:21.389490 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.389491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.389501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.389570 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.389705 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.390178 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.390927 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.391114 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.391194 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.391363 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.391405 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.391465 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.391524 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.391543 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:04:21.392334 10210 raft_consensus.cc:1261] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.392374 10069 raft_consensus.cc:1261] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Refusing update from remote peer 7fdb7effcb9c4065adc4561531ab5946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.392568 10468 consensus_queue.cc:1035] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.392673 10427 consensus_queue.cc:1035] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.395386 9912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.395467 9912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.396065 9912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Sending DeleteTablet for 3 replicas of tablet e581eb16305446a3b444b4e543f2d2a6
I20260926 13:04:21.396324 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet e581eb16305446a3b444b4e543f2d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42470
I20260926 13:04:21.396348 10190 tablet_service.cc:1514] Processing DeleteTablet for tablet e581eb16305446a3b444b4e543f2d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36724
I20260926 13:04:21.396413 10446 tablet_replica.cc:330] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: stopping tablet replica
I20260926 13:04:21.396428 10444 tablet_replica.cc:330] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: stopping tablet replica
I20260926 13:04:21.396457 10446 raft_consensus.cc:2229] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.396476 10444 raft_consensus.cc:2229] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.396493 10446 raft_consensus.cc:2258] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.396517 10444 raft_consensus.cc:2258] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.396914 10329 tablet_service.cc:1514] Processing DeleteTablet for tablet e581eb16305446a3b444b4e543f2d2a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33286
I20260926 13:04:21.396998 10446 ts_tablet_manager.cc:1905] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.397017 10445 tablet_replica.cc:330] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: stopping tablet replica
I20260926 13:04:21.397066 10445 raft_consensus.cc:2229] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:21.397121 10445 raft_consensus.cc:2258] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.398547 9912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:21.398690 9912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:21.398741 10446 ts_tablet_manager.cc:1918] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:21.398801 10446 log.cc:1196] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/e581eb16305446a3b444b4e543f2d2a6
I20260926 13:04:21.399068 10446 ts_tablet_manager.cc:1939] T e581eb16305446a3b444b4e543f2d2a6 P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting consensus metadata
I20260926 13:04:21.399113 10445 ts_tablet_manager.cc:1905] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.399374 9900 catalog_manager.cc:4841] TS bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573): tablet e581eb16305446a3b444b4e543f2d2a6 (table test-table [id=35e15b18474f402d9fe626073e2a0b78]) successfully deleted
I20260926 13:04:21.400035 10190 tablet_service.cc:1467] Processing CreateTablet for tablet 0e39377e668a4332a0c4190a64023edc (DEFAULT_TABLE table=test-table [id=c1732822ab2d463ba7659ce98fe0ef08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.400077 10329 tablet_service.cc:1467] Processing CreateTablet for tablet 0e39377e668a4332a0c4190a64023edc (DEFAULT_TABLE table=test-table [id=c1732822ab2d463ba7659ce98fe0ef08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.400141 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e39377e668a4332a0c4190a64023edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.400153 10329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e39377e668a4332a0c4190a64023edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.400372 10445 ts_tablet_manager.cc:1918] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:21.400416 10445 log.cc:1196] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/e581eb16305446a3b444b4e543f2d2a6
I20260926 13:04:21.400597 10445 ts_tablet_manager.cc:1939] T e581eb16305446a3b444b4e543f2d2a6 P 7fdb7effcb9c4065adc4561531ab5946: Deleting consensus metadata
I20260926 13:04:21.401340 10421 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Bootstrap starting.
I20260926 13:04:21.401417 10423 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Bootstrap starting.
I20260926 13:04:21.401451 10444 ts_tablet_manager.cc:1905] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.401678 10421 tablet_bootstrap.cc:654] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.401943 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 0e39377e668a4332a0c4190a64023edc (DEFAULT_TABLE table=test-table [id=c1732822ab2d463ba7659ce98fe0ef08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:21.401988 10423 tablet_bootstrap.cc:654] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.402024 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e39377e668a4332a0c4190a64023edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:21.402832 10444 ts_tablet_manager.cc:1918] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:21.402881 10444 log.cc:1196] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/e581eb16305446a3b444b4e543f2d2a6
I20260926 13:04:21.402953 10423 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: No bootstrap required, opened a new log
I20260926 13:04:21.402990 10423 ts_tablet_manager.cc:1397] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:21.403050 10422 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: Bootstrap starting.
I20260926 13:04:21.403100 10444 ts_tablet_manager.cc:1939] T e581eb16305446a3b444b4e543f2d2a6 P 38b3b51f62314e72a139f9c03e206a9c: Deleting consensus metadata
I20260926 13:04:21.403138 10423 raft_consensus.cc:348] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.403213 10423 raft_consensus.cc:374] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.403231 10423 raft_consensus.cc:729] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.403283 10423 consensus_queue.cc:260] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.403345 10422 tablet_bootstrap.cc:654] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:21.403378 10423 ts_tablet_manager.cc:1428] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.403582 9900 catalog_manager.cc:4841] TS 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589): tablet e581eb16305446a3b444b4e543f2d2a6 (table test-table [id=35e15b18474f402d9fe626073e2a0b78]) successfully deleted
I20260926 13:04:21.403784 10421 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: No bootstrap required, opened a new log
I20260926 13:04:21.403805 9898 catalog_manager.cc:4841] TS 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447): tablet e581eb16305446a3b444b4e543f2d2a6 (table test-table [id=35e15b18474f402d9fe626073e2a0b78]) successfully deleted
I20260926 13:04:21.403821 10421 ts_tablet_manager.cc:1397] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:21.403909 10421 raft_consensus.cc:348] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.403961 10421 raft_consensus.cc:374] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.403980 10421 raft_consensus.cc:729] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb7effcb9c4065adc4561531ab5946, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.404016 10421 consensus_queue.cc:260] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.404094 10421 ts_tablet_manager.cc:1428] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.404170 10422 tablet_bootstrap.cc:492] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: No bootstrap required, opened a new log
I20260926 13:04:21.404213 10422 ts_tablet_manager.cc:1397] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:04:21.404305 10422 raft_consensus.cc:348] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.404366 10422 raft_consensus.cc:374] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:21.404389 10422 raft_consensus.cc:729] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb3f23fdd2c4dd5826fc610ff55ee51, State: Initialized, Role: FOLLOWER
I20260926 13:04:21.404425 10422 consensus_queue.cc:260] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.404511 10422 ts_tablet_manager.cc:1428] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:21.805806 10428 raft_consensus.cc:482] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:21.805892 10428 raft_consensus.cc:504] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.806329 10428 leader_election.cc:288] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447)
I20260926 13:04:21.809311 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.806561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:43817 (local address 127.2.86.1:42573)
0926 13:04:21.806692 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.806694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.806702 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:21.806762 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.806763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.806773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.806863 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:21.807029 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.807514 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.808528 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.808954 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.809039 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.809052 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.809200 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.809238 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260926 13:04:21.809319 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.806560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:48473 (local address 127.2.86.3:37447)
0926 13:04:21.806608 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.806610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.806800 (+ 190us) server_negotiation.cc:407] Connection header received
0926 13:04:21.806866 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.806867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.806877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.806940 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:21.807088 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.807585 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.808545 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.808957 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.809039 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.809051 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.809219 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.809251 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:04:21.809330 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.806504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42573 (local address 127.2.86.2:43817)
0926 13:04:21.806631 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.806643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.806699 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.806865 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.806866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.806868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:21.806930 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.806932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.807599 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.807600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.808424 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.808426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.809042 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.809042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.809120 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.809124 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.809194 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.809235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260926 13:04:21.809351 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.806513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.2.86.2:48473)
0926 13:04:21.806728 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.806737 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.806814 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.806944 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.806945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.806947 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:21.807007 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.807008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.807680 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.807681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.808453 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.808455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.809041 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.809042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.809136 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.809139 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.809213 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.809251 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":49,"threads_started":1}
I20260926 13:04:21.809741 10350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e39377e668a4332a0c4190a64023edc" candidate_uuid: "38b3b51f62314e72a139f9c03e206a9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb7effcb9c4065adc4561531ab5946" is_pre_election: true
I20260926 13:04:21.809741 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e39377e668a4332a0c4190a64023edc" candidate_uuid: "38b3b51f62314e72a139f9c03e206a9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" is_pre_election: true
I20260926 13:04:21.809835 10350 raft_consensus.cc:2454] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b51f62314e72a139f9c03e206a9c in term 0.
I20260926 13:04:21.809834 10069 raft_consensus.cc:2454] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b51f62314e72a139f9c03e206a9c in term 0.
I20260926 13:04:21.810030 10144 leader_election.cc:302] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b51f62314e72a139f9c03e206a9c, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.810175 10428 raft_consensus.cc:2781] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:21.810226 10428 raft_consensus.cc:482] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:21.810246 10428 raft_consensus.cc:3037] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.810863 10428 raft_consensus.cc:504] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.811002 10428 leader_election.cc:288] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [CANDIDATE]: Term 1 election: Requested vote from peers bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573), 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447)
I20260926 13:04:21.811143 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e39377e668a4332a0c4190a64023edc" candidate_uuid: "38b3b51f62314e72a139f9c03e206a9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51"
I20260926 13:04:21.811208 10069 raft_consensus.cc:3037] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.811173 10350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e39377e668a4332a0c4190a64023edc" candidate_uuid: "38b3b51f62314e72a139f9c03e206a9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb7effcb9c4065adc4561531ab5946"
I20260926 13:04:21.811241 10350 raft_consensus.cc:3037] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:21.811848 10069 raft_consensus.cc:2454] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b51f62314e72a139f9c03e206a9c in term 1.
I20260926 13:04:21.811926 10350 raft_consensus.cc:2454] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b51f62314e72a139f9c03e206a9c in term 1.
I20260926 13:04:21.812023 10144 leader_election.cc:302] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b51f62314e72a139f9c03e206a9c, bfb3f23fdd2c4dd5826fc610ff55ee51; no voters:
I20260926 13:04:21.812127 10428 raft_consensus.cc:2781] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:21.812232 10428 raft_consensus.cc:686] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b51f62314e72a139f9c03e206a9c, State: Running, Role: LEADER
I20260926 13:04:21.812292 10428 consensus_queue.cc:237] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } }
I20260926 13:04:21.812969 9913 catalog_manager.cc:5495] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2). New cstate: current_term: 1 leader_uuid: "38b3b51f62314e72a139f9c03e206a9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b51f62314e72a139f9c03e206a9c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 46589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260926 13:04:21.846274 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.844128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.2.86.2:46589)
0926 13:04:21.844275 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.844276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.844290 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:04:21.844323 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.844323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.844340 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.844408 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.844493 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.845011 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.845651 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.845836 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.845884 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.846007 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.846106 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.846157 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.846192 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.846207 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260926 13:04:21.850540 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.848562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:50029 (local address 127.2.86.3:37447)
0926 13:04:21.848620 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.848622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.848647 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:04:21.848697 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.848698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.848704 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.848749 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.848845 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.849336 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.850001 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.850162 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.850203 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.850362 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.850385 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.850422 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.850479 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.850493 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:04:21.850565 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.848427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:37447 (local address 127.2.86.2:50029)
0926 13:04:21.848592 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.848602 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.848659 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.848762 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.848762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.848764 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:04:21.848808 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.848809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.849395 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.849395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.849928 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.849929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.850226 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.850227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.850269 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.850279 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.850434 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.850435 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.850476 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.850505 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260926 13:04:21.852586 10350 raft_consensus.cc:1261] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Refusing update from remote peer 38b3b51f62314e72a139f9c03e206a9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.852619 10069 raft_consensus.cc:1261] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Refusing update from remote peer 38b3b51f62314e72a139f9c03e206a9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:21.852842 10428 consensus_queue.cc:1035] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb3f23fdd2c4dd5826fc610ff55ee51" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.853004 10501 consensus_queue.cc:1035] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdb7effcb9c4065adc4561531ab5946" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:21.859972 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.857776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:46963 (local address 127.2.86.2:46589)
0926 13:04:21.857824 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:21.857826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:21.857915 (+ 89us) server_negotiation.cc:407] Connection header received
0926 13:04:21.857977 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:21.857978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:21.857989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:21.858034 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:21.858170 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.858590 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.859265 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.859449 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.859535 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.859684 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.859713 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.859780 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:21.859887 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 13:04:21.859907 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:04:21.859982 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.857746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:46589 (local address 127.2.86.3:46963)
0926 13:04:21.857835 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.857844 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:21.857920 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:21.858075 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:21.858076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:21.858078 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 13:04:21.858115 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.858117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.858649 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.858649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.859195 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:21.859197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:21.859546 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:21.859547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:21.859603 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:21.859609 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 13:04:21.859792 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 13:04:21.859794 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:21.859835 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 13:04:21.859879 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":53,"threads_started":1}
I20260926 13:04:21.866582 9913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.866675 9913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:21.867277 9913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49d5bee5972f4d41abdfb63873e5b9fc: Sending DeleteTablet for 3 replicas of tablet 0e39377e668a4332a0c4190a64023edc
I20260926 13:04:21.867506 10329 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e39377e668a4332a0c4190a64023edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33286
I20260926 13:04:21.867600 10445 tablet_replica.cc:330] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: stopping tablet replica
I20260926 13:04:21.867588 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e39377e668a4332a0c4190a64023edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42470
I20260926 13:04:21.867621 10190 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e39377e668a4332a0c4190a64023edc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:21 UTC) from {username='oryx', principal='oryx/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:36724
I20260926 13:04:21.867645 10445 raft_consensus.cc:2229] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.867666 10446 tablet_replica.cc:330] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: stopping tablet replica
I20260926 13:04:21.867677 10445 raft_consensus.cc:2258] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.867697 10444 tablet_replica.cc:330] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: stopping tablet replica
I20260926 13:04:21.867710 10446 raft_consensus.cc:2229] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:21.867734 10444 raft_consensus.cc:2229] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:21.867743 10446 raft_consensus.cc:2258] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.867796 10444 raft_consensus.cc:2258] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:21.868048 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9985
I20260926 13:04:21.868104 10446 ts_tablet_manager.cc:1905] T 0e39377e668a4332a0c4190a64023edc P bfb3f23fdd2c4dd5826fc610ff55ee51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.868999 10445 ts_tablet_manager.cc:1905] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.870543 10445 ts_tablet_manager.cc:1918] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:21.870616 10445 log.cc:1196] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/0e39377e668a4332a0c4190a64023edc
I20260926 13:04:21.870678 10444 ts_tablet_manager.cc:1905] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:21.870889 10445 ts_tablet_manager.cc:1939] T 0e39377e668a4332a0c4190a64023edc P 7fdb7effcb9c4065adc4561531ab5946: Deleting consensus metadata
I20260926 13:04:21.871165 9898 catalog_manager.cc:4841] TS 7fdb7effcb9c4065adc4561531ab5946 (127.2.86.3:37447): tablet 0e39377e668a4332a0c4190a64023edc (table test-table [id=c1732822ab2d463ba7659ce98fe0ef08]) successfully deleted
I20260926 13:04:21.872125 10444 ts_tablet_manager.cc:1918] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 13:04:21.872187 10444 log.cc:1196] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/0e39377e668a4332a0c4190a64023edc
I20260926 13:04:21.872388 10444 ts_tablet_manager.cc:1939] T 0e39377e668a4332a0c4190a64023edc P 38b3b51f62314e72a139f9c03e206a9c: Deleting consensus metadata
I20260926 13:04:21.872946 9900 catalog_manager.cc:4841] TS 38b3b51f62314e72a139f9c03e206a9c (127.2.86.2:46589): tablet 0e39377e668a4332a0c4190a64023edc (table test-table [id=c1732822ab2d463ba7659ce98fe0ef08]) successfully deleted
I20260926 13:04:21.878000 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10125
W20260926 13:04:21.879829 9900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:42573 (error 108)
I20260926 13:04:21.880468 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:21.880233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42573 (local address 127.0.0.1:45784)
0926 13:04:21.880347 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:21.880390 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:42573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":44,"threads_started":1}
W20260926 13:04:21.880611 9900 catalog_manager.cc:4584] TS bfb3f23fdd2c4dd5826fc610ff55ee51 (127.2.86.1:42573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e39377e668a4332a0c4190a64023edc: Network error: Client connection negotiation failed: client connection to 127.2.86.1:42573: connect: Connection refused (error 111)
I20260926 13:04:21.884310 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10265
I20260926 13:04:21.891127 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 9881
2026-09-26T13:04:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3017 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[10518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[10518](info): set up 2 sockets
Sep 26 13:04:21 dist-test-slave-j4lq krb5kdc[10518](info): commencing operation
krb5kdc: starting...
W20260926 13:04:22.913465 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:22 dist-test-slave-j4lq krb5kdc[10518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:23.027513 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42567
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:23.105499 10531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:23.105713 10531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:23.105764 10531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:23.106922 10531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:23.106988 10531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:23.107012 10531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:23.107033 10531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:23.107061 10531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:23.108562 10531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:23.108855 10531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:23.109095 10531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:23.113516 10539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:23.114930 10531 server_base.cc:1034] running on GCE node
W20260926 13:04:23.115025 10536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:23.115049 10537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:23.115316 10531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:23.115550 10531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:23.116689 10531 hybrid_clock.cc:648] HybridClock initialized: now 1790427863116682 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T13:04:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1233 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:23 dist-test-slave-j4lq krb5kdc[10544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:23 dist-test-slave-j4lq krb5kdc[10544](info): set up 2 sockets
Sep 26 13:04:23 dist-test-slave-j4lq krb5kdc[10544](info): commencing operation
krb5kdc: starting...
W20260926 13:04:24.153400 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:24.269097 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39575
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:24.346736 10557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:24.346928 10557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:24.346949 10557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:24.348111 10557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:24.348151 10557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:24.348165 10557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:24.348178 10557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:24.348229 10557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:24.349690 10557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:24.349942 10557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:24.350162 10557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:24.354775 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.354759 10562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.354729 10565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:24.354930 10557 server_base.cc:1034] running on GCE node
I20260926 13:04:24.355213 10557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:24.355473 10557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:24.356611 10557 hybrid_clock.cc:648] HybridClock initialized: now 1790427864356601 us; error 28 us; skew 500 ppm
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:24.366029 10557 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:24.366438 10557 webserver.cc:460] Webserver started at http://127.2.86.62:42645/ using document root <none> and password file <none>
I20260926 13:04:24.366667 10557 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:24.366711 10557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:24.366853 10557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:24.367864 10557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "f5ad58df06f348fbaa09210fb68b5c83"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "46ab4e34e18c465401df1a4aa86e141b"
server_key_iv: "0d9ae577b0e202a3e4b00ea9c47a890d"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.368213 10557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "f5ad58df06f348fbaa09210fb68b5c83"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "46ab4e34e18c465401df1a4aa86e141b"
server_key_iv: "0d9ae577b0e202a3e4b00ea9c47a890d"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.369539 10557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:24.370255 10572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.370435 10557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:24.370505 10557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "f5ad58df06f348fbaa09210fb68b5c83"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "46ab4e34e18c465401df1a4aa86e141b"
server_key_iv: "0d9ae577b0e202a3e4b00ea9c47a890d"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.370595 10557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:24.388405 10557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:24.390241 10557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:24.390386 10557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:24.394277 10557 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:39575
I20260926 13:04:24.394341 10624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:39575 every 8 connection(s)
I20260926 13:04:24.394660 10557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:24.395462 10625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:24.397644 10625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Bootstrap starting.
I20260926 13:04:24.398137 10625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:24.398360 10625 log.cc:824] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:24.398937 10625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: No bootstrap required, opened a new log
I20260926 13:04:24.400368 10625 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:24.400491 10625 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:24.400512 10625 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ad58df06f348fbaa09210fb68b5c83, State: Initialized, Role: FOLLOWER
I20260926 13:04:24.400604 10625 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:24.400650 10625 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:24.400669 10625 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:24.400698 10625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:24.401206 10625 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:24.401283 10625 leader_election.cc:302] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ad58df06f348fbaa09210fb68b5c83; no voters:
I20260926 13:04:24.401392 10625 leader_election.cc:288] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:24.401467 10628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:24.401630 10625 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:24.401815 10628 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 LEADER]: Becoming Leader. State: Replica: f5ad58df06f348fbaa09210fb68b5c83, State: Running, Role: LEADER
I20260926 13:04:24.401911 10628 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:24.402297 10630 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ad58df06f348fbaa09210fb68b5c83. Latest consensus state: current_term: 1 leader_uuid: "f5ad58df06f348fbaa09210fb68b5c83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } } }
I20260926 13:04:24.402308 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ad58df06f348fbaa09210fb68b5c83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } } }
I20260926 13:04:24.402379 10630 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:24.402444 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:24.402616 10637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:24.403126 10637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:24.403316 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10557
I20260926 13:04:24.403429 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:24.403633 2392 external_mini_cluster.cc:1428] Setting key 6c81641ecba66c7e2bf5306082443e31
I20260926 13:04:24.405267 10637 catalog_manager.cc:1347] Generated new cluster ID: 12101af0547046c19ee8361bdffc1a7d
I20260926 13:04:24.405319 10637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:24.414784 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.404014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.2.86.62:39575)
0926 13:04:24.404293 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:24.404302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:24.404332 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:04:24.404433 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:24.404438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:24.404487 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:24.404618 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:24.405403 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.406840 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.407865 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.408488 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.408603 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.411308 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:24.411311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:24.411311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:24.411322 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:24.413180 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.413573 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.413575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.413576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.413668 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.413890 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.413891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.413891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.414164 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:24.414302 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:24.414427 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 13:04:24.414547 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260926 13:04:24.425989 10637 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:24.426467 10637 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:24.440491 10637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Generated new TSK 0
I20260926 13:04:24.440683 10637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:24.466073 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:24.545977 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:24.546157 10651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:24.546178 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:24.547281 10651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:24.547319 10651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:24.547369 10651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:24.548748 10651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:24.549010 10651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:24.549221 10651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:24.553655 10656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.553668 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.553670 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:24.553891 10651 server_base.cc:1034] running on GCE node
I20260926 13:04:24.554162 10651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:24.554351 10651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:24.555531 10651 hybrid_clock.cc:648] HybridClock initialized: now 1790427864555515 us; error 45 us; skew 500 ppm
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:24.557979 10651 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:24.558373 10651 webserver.cc:460] Webserver started at http://127.2.86.1:41295/ using document root <none> and password file <none>
I20260926 13:04:24.558606 10651 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:24.558672 10651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:24.558818 10651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:24.559779 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "9d4e868109ba4a68953aece4ebaf3a25"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "1e0cd67efd066f1e69f726a7cdd4d502"
server_key_iv: "866d291f3a5b4733121a2874a05f04af"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.560119 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "9d4e868109ba4a68953aece4ebaf3a25"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "1e0cd67efd066f1e69f726a7cdd4d502"
server_key_iv: "866d291f3a5b4733121a2874a05f04af"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.561432 10651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:24.562108 10666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.562266 10651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:24.562350 10651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "9d4e868109ba4a68953aece4ebaf3a25"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "1e0cd67efd066f1e69f726a7cdd4d502"
server_key_iv: "866d291f3a5b4733121a2874a05f04af"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.562423 10651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:24.580713 10651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:24.582463 10651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:24.582629 10651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:24.582849 10651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:24.583148 10651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:24.583201 10651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.583235 10651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:24.583263 10651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.588788 10651 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41569
I20260926 13:04:24.588860 10779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41569 every 8 connection(s)
I20260926 13:04:24.589202 10651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:24.590621 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10651
I20260926 13:04:24.590730 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:24.590885 2392 external_mini_cluster.cc:1428] Setting key 3426fc54d72c453443dd0c8de7feff28
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:24.599277 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.589868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:38175)
0926 13:04:24.590026 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:24.590059 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:24.590533 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:24.590813 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:24.590816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:24.591539 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:24.591762 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.591766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.592767 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.592768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.593596 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.593599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.594073 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.594073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.594165 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.594650 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:24.594658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:24.596178 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:24.598059 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.598065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.598066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.598263 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.598465 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.598466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.598466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.598554 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.598821 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:24.598825 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:24.598930 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 13:04:24.599015 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 13:04:24.599126 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260926 13:04:24.599294 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.589875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:38175 (local address 127.2.86.62:39575)
0926 13:04:24.589943 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:24.589945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:24.590168 (+ 223us) server_negotiation.cc:407] Connection header received
0926 13:04:24.590635 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:24.590637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:24.590649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:24.590723 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:24.591859 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.592679 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.593680 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.593999 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.594058 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.596272 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:24.596273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:24.596274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:24.596281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:24.597971 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.598334 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.598336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.598336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.598397 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.598621 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.598622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.598623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.598751 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:24.598829 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:24.599064 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 13:04:24.599159 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
I20260926 13:04:24.599787 10780 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39575
I20260926 13:04:24.599877 10780 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:24.600080 10780 heartbeater.cc:511] Master 127.2.86.62:39575 requested a full tablet report, sending...
I20260926 13:04:24.600546 10589 ts_manager.cc:194] Registered new tserver with Master: 9d4e868109ba4a68953aece4ebaf3a25 (127.2.86.1:41569)
I20260926 13:04:24.601238 10589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:38175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:24.615931 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:24.694546 10785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:24.694743 10785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:24.694764 10785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:24.695919 10785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:24.695957 10785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:24.695983 10785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:24.697435 10785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:24.697672 10785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:24.697885 10785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:24.702917 10793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.702924 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:24.702960 10785 server_base.cc:1034] running on GCE node
W20260926 13:04:24.703111 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:24.703476 10785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:24.703739 10785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:24.704928 10785 hybrid_clock.cc:648] HybridClock initialized: now 1790427864704912 us; error 28 us; skew 500 ppm
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:24.707405 10785 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:24.707861 10785 webserver.cc:460] Webserver started at http://127.2.86.2:42413/ using document root <none> and password file <none>
I20260926 13:04:24.708087 10785 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:24.708154 10785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:24.708302 10785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:24.709445 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "738497ef499f46c1afd88b37902cf2c5"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "a6b592e1b0defa776c336020daff8020"
server_key_iv: "4796e1880170ede09bbdc5447a7a3742"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.709801 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "738497ef499f46c1afd88b37902cf2c5"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "a6b592e1b0defa776c336020daff8020"
server_key_iv: "4796e1880170ede09bbdc5447a7a3742"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.711050 10785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:24.711778 10800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.711953 10785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:24.712049 10785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "738497ef499f46c1afd88b37902cf2c5"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "a6b592e1b0defa776c336020daff8020"
server_key_iv: "4796e1880170ede09bbdc5447a7a3742"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.712132 10785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:24.729784 10785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:24.732168 10785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:24.732338 10785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:24.732584 10785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:24.732972 10785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:24.733026 10785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.733068 10785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:24.733095 10785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.739081 10785 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:39715
I20260926 13:04:24.739113 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:39715 every 8 connection(s)
I20260926 13:04:24.739449 10785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:24.740645 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10785
I20260926 13:04:24.740756 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:24.740967 2392 external_mini_cluster.cc:1428] Setting key 8c9fb8cb9af4d05d46194a0af0d5aa0a
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
I20260926 13:04:24.749560 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.740189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51205 (local address 127.2.86.62:39575)
0926 13:04:24.740251 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:24.740254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:24.740435 (+ 181us) server_negotiation.cc:407] Connection header received
0926 13:04:24.741128 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:24.741130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:24.741148 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:24.741204 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:24.742274 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.742771 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.743888 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.744113 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.744193 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.746657 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:24.746659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:24.746659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:24.746667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:24.748097 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.748521 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.748523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.748523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.748586 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.748846 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.748847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.748847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.749003 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:24.749088 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:24.749390 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 13:04:24.749471 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:04:24.749771 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.740131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:51205)
0926 13:04:24.740297 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:24.740340 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:24.740974 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:24.741278 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:24.741282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:24.741967 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:24.742159 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.742164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.742899 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.742908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.743771 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.743774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.744213 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.744214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.744342 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.744957 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:24.744979 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:24.746560 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:24.748198 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.748205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.748207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.748430 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.748652 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.748653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.748653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.748759 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.749078 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:24.749085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:24.749232 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 13:04:24.749369 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 13:04:24.749524 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260926 13:04:24.750367 10914 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39575
I20260926 13:04:24.750476 10914 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:24.750703 10914 heartbeater.cc:511] Master 127.2.86.62:39575 requested a full tablet report, sending...
I20260926 13:04:24.751147 10589 ts_manager.cc:194] Registered new tserver with Master: 738497ef499f46c1afd88b37902cf2c5 (127.2.86.2:39715)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
I20260926 13:04:24.751597 10589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:51205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:24.765494 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:24.844444 10919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:24.844630 10919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:24.844651 10919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:24.845789 10919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:24.845826 10919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:24.845852 10919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:24.847153 10919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:24.847312 10919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:24.847466 10919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:24.852077 10927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.852095 10924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:24.852105 10925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:24.852124 10919 server_base.cc:1034] running on GCE node
I20260926 13:04:24.852409 10919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:24.852655 10919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:24.853819 10919 hybrid_clock.cc:648] HybridClock initialized: now 1790427864853808 us; error 37 us; skew 500 ppm
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:24.856163 10919 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:24.856591 10919 webserver.cc:460] Webserver started at http://127.2.86.3:33579/ using document root <none> and password file <none>
I20260926 13:04:24.856825 10919 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:24.856918 10919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:24.857069 10919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:24.858131 10919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "a1f7b68f9c8b47e8aa7cbb2ce255fcef"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "41dc7eaa054294de8fd35e38bc521a99"
server_key_iv: "a4f6e134ae6be9fca56d2a6bb6c66c0f"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.858481 10919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "a1f7b68f9c8b47e8aa7cbb2ce255fcef"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "41dc7eaa054294de8fd35e38bc521a99"
server_key_iv: "a4f6e134ae6be9fca56d2a6bb6c66c0f"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.859697 10919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:24.860354 10934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.860517 10919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:24.860612 10919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a1f7b68f9c8b47e8aa7cbb2ce255fcef"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "41dc7eaa054294de8fd35e38bc521a99"
server_key_iv: "a4f6e134ae6be9fca56d2a6bb6c66c0f"
server_key_version: "encryptionkey@0"
I20260926 13:04:24.860688 10919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:24.879518 10919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:24.881258 10919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:24.881403 10919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:24.881614 10919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:24.881891 10919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:24.881942 10919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.881987 10919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:24.882014 10919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:24.887521 10919 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:44307
I20260926 13:04:24.887560 11047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:44307 every 8 connection(s)
I20260926 13:04:24.887888 10919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:24.890004 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 10919
I20260926 13:04:24.890123 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:24.890331 2392 external_mini_cluster.cc:1428] Setting key 6bf654802f68bef4a5f97412967830b3
Sep 26 13:04:24 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:24.896585 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.888592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:51507 (local address 127.2.86.62:39575)
0926 13:04:24.888648 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:24.888650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:24.888827 (+ 177us) server_negotiation.cc:407] Connection header received
0926 13:04:24.889283 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:24.889284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:24.889293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:24.889366 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:24.890345 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.891124 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.892132 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.892302 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.892353 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.894297 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:24.894298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:24.894298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:24.894303 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:24.895491 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.895831 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.895832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.895832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.895881 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:24.896101 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:24.896102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:24.896102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:24.896197 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:24.896244 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:24.896465 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 13:04:24.896522 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:04:24.896741 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:24.888543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:51507)
0926 13:04:24.888701 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:24.888740 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:24.889207 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:24.889437 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:24.889441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:24.889979 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:24.890221 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.890246 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.891211 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.891212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.892049 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:24.892052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:24.892373 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:24.892374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:24.892466 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:24.892977 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:24.892985 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:24.894224 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:24.895560 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.895565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.895567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.895768 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.895938 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:24.895939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:24.895939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:24.896034 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:24.896249 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:24.896252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:24.896359 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 13:04:24.896438 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 13:04:24.896549 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260926 13:04:24.897244 11048 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39575
I20260926 13:04:24.897344 11048 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:24.897557 11048 heartbeater.cc:511] Master 127.2.86.62:39575 requested a full tablet report, sending...
I20260926 13:04:24.897941 10589 ts_manager.cc:194] Registered new tserver with Master: a1f7b68f9c8b47e8aa7cbb2ce255fcef (127.2.86.3:44307)
I20260926 13:04:24.898521 10589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:51507
I20260926 13:04:24.901890 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
Entry for principal oryx/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab.
WARNING: no policy specified for oryx/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
Entry for principal oryx/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab.
WARNING: no policy specified for oryx/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
Entry for principal oryx/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab.
I20260926 13:04:24.967056 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10651
I20260926 13:04:24.971798 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10785
I20260926 13:04:24.976263 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10919
I20260926 13:04:24.980582 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 10557
I20260926 13:04:24.985692 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39575
--webserver_interface=127.2.86.62
--webserver_port=42645
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:25.065404 11057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:25.065630 11057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:25.065665 11057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:25.066792 11057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:25.066862 11057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:25.066886 11057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:25.066915 11057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:25.066941 11057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:25.068382 11057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=42645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:25.068645 11057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:25.068919 11057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:25.073657 11063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.073686 11065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:25.073678 11057 server_base.cc:1034] running on GCE node
W20260926 13:04:25.073662 11062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:25.074138 11057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:25.074427 11057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:25.075568 11057 hybrid_clock.cc:648] HybridClock initialized: now 1790427865075554 us; error 25 us; skew 500 ppm
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427865, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:25.078071 11057 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:25.078516 11057 webserver.cc:460] Webserver started at http://127.2.86.62:42645/ using document root <none> and password file <none>
I20260926 13:04:25.078744 11057 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:25.078817 11057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:25.080113 11057 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:25.080924 11072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.081086 11057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:25.081183 11057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "f5ad58df06f348fbaa09210fb68b5c83"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "46ab4e34e18c465401df1a4aa86e141b"
server_key_iv: "0d9ae577b0e202a3e4b00ea9c47a890d"
server_key_version: "encryptionkey@0"
I20260926 13:04:25.081522 11057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:25.103191 11057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:25.105636 11057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:25.105813 11057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:25.109910 11057 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:39575
I20260926 13:04:25.109957 11124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:39575 every 8 connection(s)
I20260926 13:04:25.110342 11057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:25.111714 11125 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:04:25.112280 11125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Bootstrap starting.
I20260926 13:04:25.120301 11125 log.cc:824] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:25.120378 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11057
I20260926 13:04:25.120692 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:41569
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:39575
--webserver_port=41295
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:04:25.122234 11125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:04:25.122498 11125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Bootstrap complete.
I20260926 13:04:25.124744 11125 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:25.125008 11125 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ad58df06f348fbaa09210fb68b5c83, State: Initialized, Role: FOLLOWER
I20260926 13:04:25.125165 11125 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:25.125231 11125 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:25.125288 11125 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:25.125331 11125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:04:25.126222 11125 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:25.126298 11125 leader_election.cc:302] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ad58df06f348fbaa09210fb68b5c83; no voters:
I20260926 13:04:25.126456 11125 leader_election.cc:288] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:04:25.126514 11129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:04:25.126629 11129 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [term 2 LEADER]: Becoming Leader. State: Replica: f5ad58df06f348fbaa09210fb68b5c83, State: Running, Role: LEADER
I20260926 13:04:25.126696 11125 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:25.126708 11129 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } }
I20260926 13:04:25.127722 11131 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ad58df06f348fbaa09210fb68b5c83. Latest consensus state: current_term: 2 leader_uuid: "f5ad58df06f348fbaa09210fb68b5c83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } } }
I20260926 13:04:25.127810 11131 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:25.127895 11130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f5ad58df06f348fbaa09210fb68b5c83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ad58df06f348fbaa09210fb68b5c83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39575 } } }
I20260926 13:04:25.127985 11130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:25.128000 11140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:25.128515 11140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:25.128633 11140 catalog_manager.cc:1259] Loaded cluster ID: 12101af0547046c19ee8361bdffc1a7d
I20260926 13:04:25.128667 11140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:25.129823 11140 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:25.129992 11140 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f5ad58df06f348fbaa09210fb68b5c83: Loaded TSK: 0
I20260926 13:04:25.130200 11140 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:04:25.201143 11128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:25.201359 11128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:25.201395 11128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:25.202597 11128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:25.202668 11128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:25.202715 11128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:25.204205 11128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.1.keytab
--principal=oryx/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:41569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=41295
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:25.204468 11128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:25.204697 11128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:25.209707 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.209709 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.209707 11154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:25.210530 11128 server_base.cc:1034] running on GCE node
I20260926 13:04:25.210726 11128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:25.210973 11128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:25.212131 11128 hybrid_clock.cc:648] HybridClock initialized: now 1790427865212111 us; error 43 us; skew 500 ppm
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427865, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:25.214915 11128 init.cc:377] Logged in from keytab as oryx/127.2.86.1@KRBTEST.COM (short username oryx)
I20260926 13:04:25.215482 11128 webserver.cc:460] Webserver started at http://127.2.86.1:41295/ using document root <none> and password file <none>
I20260926 13:04:25.215759 11128 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:25.215839 11128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:25.217630 11128 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:25.218263 11161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.218492 11128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:25.218578 11128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "9d4e868109ba4a68953aece4ebaf3a25"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "1e0cd67efd066f1e69f726a7cdd4d502"
server_key_iv: "866d291f3a5b4733121a2874a05f04af"
server_key_version: "encryptionkey@0"
I20260926 13:04:25.219008 11128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:25.240262 11128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:25.242630 11128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:25.242774 11128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:25.243017 11128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:25.243325 11128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:25.243362 11128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.243391 11128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:25.243417 11128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.250324 11128 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41569
I20260926 13:04:25.250361 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41569 every 8 connection(s)
I20260926 13:04:25.250787 11128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:25.255707 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11128
I20260926 13:04:25.256052 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:39715
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:39575
--webserver_port=42413
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.261682 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.251553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:56615)
0926 13:04:25.251715 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.251757 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.252245 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.253515 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.253518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.254080 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.254263 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.254268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.255936 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.255938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.257305 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.257308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.257924 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.257925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.258051 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.258733 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.258744 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.261500 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
W20260926 13:04:25.261897 11275 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:39575 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.86.62:39575: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
W20260926 13:04:25.262176 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.251645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:56615 (local address 127.2.86.62:39575)
0926 13:04:25.253018 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.253031 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.253084 (+ 53us) server_negotiation.cc:407] Connection header received
0926 13:04:25.253240 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.253249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.253311 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.253493 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.254391 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.255820 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.257416 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.257826 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.257984 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.261662 (+ 3678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:56615: BlockingRecv error: recv got EOF from 127.2.86.1:56615 (error 108)
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":81,"threads_started":1}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.267261 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.262131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:60521)
0926 13:04:25.262455 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.262476 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.262531 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.262670 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.262671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.262816 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.262868 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.262870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.263844 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.263845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.264333 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.264335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.264828 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.264828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.264916 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.265410 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.265415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.267217 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306}
W20260926 13:04:25.267400 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.262185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:60521 (local address 127.2.86.62:39575)
0926 13:04:25.262331 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.262333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.262541 (+ 208us) server_negotiation.cc:407] Connection header received
0926 13:04:25.262573 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.262574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.262591 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.262668 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.262954 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.263758 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.264414 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.264749 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.264828 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.267333 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:60521: BlockingRecv error: recv got EOF from 127.2.86.1:60521 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.271790 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.267541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:38875)
0926 13:04:25.267580 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.267589 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.267630 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.267812 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.267813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.267912 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.267936 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.267937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.268795 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.268795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.269298 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.269300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.269718 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.269719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.269752 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.270199 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.270203 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.271750 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 13:04:25.271895 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.267630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:38875 (local address 127.2.86.62:39575)
0926 13:04:25.267694 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.267696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.267704 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:25.267728 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.267729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.267739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.267804 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.268002 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.268722 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.269362 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.269651 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.269716 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.271841 (+ 2125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:38875: BlockingRecv error: recv got EOF from 127.2.86.1:38875 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 13:04:25.271942 11275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:04:25.339654 11279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:25.339854 11279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:25.339876 11279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:25.341063 11279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:25.341101 11279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:25.341126 11279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:25.342525 11279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.2.keytab
--principal=oryx/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:39715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=42413
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:25.342756 11279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:25.342981 11279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:25.347852 11286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.347846 11285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.347873 11288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:25.347885 11279 server_base.cc:1034] running on GCE node
I20260926 13:04:25.348235 11279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:25.348471 11279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:25.349638 11279 hybrid_clock.cc:648] HybridClock initialized: now 1790427865349624 us; error 40 us; skew 500 ppm
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427865, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:25.351961 11279 init.cc:377] Logged in from keytab as oryx/127.2.86.2@KRBTEST.COM (short username oryx)
I20260926 13:04:25.352377 11279 webserver.cc:460] Webserver started at http://127.2.86.2:42413/ using document root <none> and password file <none>
I20260926 13:04:25.352566 11279 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:25.352615 11279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:25.353935 11279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:25.354444 11295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.354614 11279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:25.354692 11279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "738497ef499f46c1afd88b37902cf2c5"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "a6b592e1b0defa776c336020daff8020"
server_key_iv: "4796e1880170ede09bbdc5447a7a3742"
server_key_version: "encryptionkey@0"
I20260926 13:04:25.354978 11279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:25.379745 11279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:25.381661 11279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:25.381785 11279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:25.381968 11279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:25.382289 11279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:25.382323 11279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.382388 11279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:25.382429 11279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.388458 11279 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:39715
I20260926 13:04:25.388538 11408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:39715 every 8 connection(s)
I20260926 13:04:25.388902 11279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:25.391883 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11279
I20260926 13:04:25.392220 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:44307
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:39575
--webserver_port=33579
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:04:25.396736 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.389635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:52081 (local address 127.2.86.62:39575)
0926 13:04:25.389684 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.389687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.389943 (+ 256us) server_negotiation.cc:407] Connection header received
0926 13:04:25.390425 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.390426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.390438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.390488 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.391346 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.391849 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.393111 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.393323 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.393397 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.396690 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:52081: BlockingRecv error: recv got EOF from 127.2.86.2:52081 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:04:25.396924 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.389639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:52081)
0926 13:04:25.389809 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.389845 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.390338 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.390541 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.390544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.391074 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.391259 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.391264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.391952 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.391953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.393014 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.393016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.393410 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.393411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.393532 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.394099 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.394107 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.396616 (+ 2509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260926 13:04:25.397199 11409 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:39575 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.86.62:39575: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.401568 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.397444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:46423)
0926 13:04:25.397478 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.397488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.397527 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.397650 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.397650 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.397779 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.397819 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.397821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.398449 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.398450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.398955 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.398956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.399317 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.399318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.399382 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.399855 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.399859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.401488 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 13:04:25.401623 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.397492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:46423 (local address 127.2.86.62:39575)
0926 13:04:25.397538 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.397540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.397549 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:25.397575 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.397576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.397586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.397646 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.397901 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.398351 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.399037 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.399234 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.399316 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.401558 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:46423: BlockingRecv error: recv got EOF from 127.2.86.2:46423 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.406407 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.402071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:50849)
0926 13:04:25.402124 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.402135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.402189 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.402308 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.402309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.402444 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.402474 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.402476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.403027 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.403028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.403836 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.403839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.404145 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.404146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.404200 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.404544 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.404548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.406338 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 13:04:25.406488 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.402114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:50849 (local address 127.2.86.62:39575)
0926 13:04:25.402148 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.402150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.402175 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:04:25.402243 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.402244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.402251 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.402294 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.402534 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.402959 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.403917 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.404080 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.404125 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.406429 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:50849: BlockingRecv error: recv got EOF from 127.2.86.2:50849 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 13:04:25.406752 11409 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:04:25.477641 11412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:25.477851 11412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:25.477874 11412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:25.479007 11412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:25.479065 11412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:25.479094 11412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:25.480458 11412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/krb5kdc/oryx_127.2.86.3.keytab
--principal=oryx/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:44307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=33579
--tserver_master_addrs=127.2.86.62:39575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:25.480636 11412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:25.480813 11412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:25.485615 11420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.485592 11417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:25.485602 11418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:25.485901 11412 server_base.cc:1034] running on GCE node
I20260926 13:04:25.486116 11412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:25.486371 11412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:25.487517 11412 hybrid_clock.cc:648] HybridClock initialized: now 1790427865487509 us; error 33 us; skew 500 ppm
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427865, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:25.490065 11412 init.cc:377] Logged in from keytab as oryx/127.2.86.3@KRBTEST.COM (short username oryx)
I20260926 13:04:25.490485 11412 webserver.cc:460] Webserver started at http://127.2.86.3:33579/ using document root <none> and password file <none>
I20260926 13:04:25.490705 11412 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:25.490782 11412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:25.492094 11412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:25.492671 11427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.492825 11412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:25.492961 11412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a1f7b68f9c8b47e8aa7cbb2ce255fcef"
format_stamp: "Formatted at 2026-09-26 13:04:24 on dist-test-slave-j4lq"
server_key: "41dc7eaa054294de8fd35e38bc521a99"
server_key_iv: "a4f6e134ae6be9fca56d2a6bb6c66c0f"
server_key_version: "encryptionkey@0"
I20260926 13:04:25.493309 11412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:25.501696 11412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:25.503989 11412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:25.504146 11412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:25.504379 11412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:25.504683 11412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:25.504734 11412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.504776 11412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:25.504803 11412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:25.510746 11412 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:44307
I20260926 13:04:25.510785 11540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:44307 every 8 connection(s)
I20260926 13:04:25.511126 11412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.517237 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11412
W20260926 13:04:25.518165 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.511836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:43139 (local address 127.2.86.62:39575)
0926 13:04:25.511909 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.511911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.512106 (+ 195us) server_negotiation.cc:407] Connection header received
0926 13:04:25.512562 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.512563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.512573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.512621 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.513580 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.514059 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.515003 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.515183 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.515234 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.518106 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:43139: BlockingRecv error: recv got EOF from 127.2.86.3:43139 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:04:25.518247 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.511828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:43139)
0926 13:04:25.511986 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.512020 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.512480 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.512681 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.512685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.513298 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.513497 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.513502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.514136 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.514136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.514925 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.514927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.515253 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.515254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.515346 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.515820 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.515828 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.518050 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
W20260926 13:04:25.518460 11541 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:39575 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.86.62:39575: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.522884 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.518684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:34025)
0926 13:04:25.518714 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.518724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.518759 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.518888 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.518889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.519012 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.519050 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.519051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.519645 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.519646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.520144 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.520145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.520515 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.520515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.520581 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.521092 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.521097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.522800 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260926 13:04:25.522960 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.518739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34025 (local address 127.2.86.62:39575)
0926 13:04:25.518783 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.518785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.518804 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:04:25.518822 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.518823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.518831 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.518880 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.519130 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.519571 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.520241 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.520434 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.520514 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.522885 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:34025: BlockingRecv error: recv got EOF from 127.2.86.3:34025 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:04:25.524991 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.517784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42046 (local address 127.2.86.62:39575)
0926 13:04:25.517833 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.517835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.517941 (+ 106us) server_negotiation.cc:407] Connection header received
0926 13:04:25.518008 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.518009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.518018 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.518095 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.518853 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.519331 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.520095 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.520301 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.520384 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.521580 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:25.521584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:25.521585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:25.521596 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:25.523531 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:25.523956 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:25.523959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:25.523959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:25.524047 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:25.524239 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:25.524240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:25.524240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:25.524434 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:25.524596 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:25.524689 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:04:25.524902 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:25 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:25.527733 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:25.523266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:34071)
0926 13:04:25.523373 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:25.523384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:25.523439 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:25.523556 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:25.523557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:25.523690 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:25.523719 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.523720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.524509 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.524510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.525283 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:25.525285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.525731 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.525732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:25.525784 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.526086 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:25.526089 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:25.527692 (+ 1603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260926 13:04:25.527832 11278 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:25.523328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34071 (local address 127.2.86.62:39575)
0926 13:04:25.523378 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:25.523380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:25.523434 (+ 54us) server_negotiation.cc:407] Connection header received
0926 13:04:25.523496 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:25.523496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:25.523503 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:25.523541 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:25.523799 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.524436 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.525376 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:25.525615 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:25.525691 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:25.527771 (+ 2080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:34071: BlockingRecv error: recv got EOF from 127.2.86.3:34071 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 13:04:25.527879 11541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:26.276530 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:26.272306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:34525)
0926 13:04:26.272439 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:26.272451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:26.272507 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:26.272694 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:26.272695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:26.272857 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:26.272949 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.272955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.273584 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.273585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.274118 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.274120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.274427 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.274428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.274464 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.274866 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:26.274870 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:26.276464 (+ 1594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
W20260926 13:04:26.276631 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:26.272359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:34525 (local address 127.2.86.62:39575)
0926 13:04:26.272481 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:26.272483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:26.272490 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:26.272576 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:26.272577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:26.272591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:26.272669 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:26.273026 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.273520 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.274189 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.274367 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.274421 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.276555 (+ 2134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:34525: BlockingRecv error: recv got EOF from 127.2.86.1:34525 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:26.411856 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:26.407311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:45173)
0926 13:04:26.407424 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:26.407437 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:26.407495 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:26.407602 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:26.407603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:26.407832 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:26.407889 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.407891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.408514 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.408515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.409123 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.409124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.409412 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.409413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.409446 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.409930 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:26.409934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:26.411751 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
W20260926 13:04:26.411882 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:26.407386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:45173 (local address 127.2.86.62:39575)
0926 13:04:26.407437 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:26.407439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:26.407481 (+ 42us) server_negotiation.cc:407] Connection header received
0926 13:04:26.407536 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:26.407536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:26.407546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:26.407597 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:26.407965 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.408439 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.409183 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.409356 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.409405 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.411821 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:45173: BlockingRecv error: recv got EOF from 127.2.86.2:45173 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:26 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:26.533092 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:26.528185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:59149)
0926 13:04:26.528326 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:26.528337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:26.528433 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:26.528599 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:26.528600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:26.528773 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:26.528815 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.528817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.529504 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.529505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.530078 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:26.530080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.530420 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.530421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:26.530487 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.531084 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:26.531089 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:26.533026 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":48,"threads_started":1}
W20260926 13:04:26.533200 11546 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:26.528253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:59149 (local address 127.2.86.62:39575)
0926 13:04:26.528326 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:26.528328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:26.528425 (+ 97us) server_negotiation.cc:407] Connection header received
0926 13:04:26.528515 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:26.528520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:26.528531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:26.528590 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:26.528919 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.529416 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.530159 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:26.530341 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:26.530421 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:26.533107 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:59149: BlockingRecv error: recv got EOF from 127.2.86.3:59149 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:27.281358 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:27.277071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:41419)
0926 13:04:27.277211 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:27.277224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:27.277292 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:27.277482 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:27.277483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:27.277645 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:27.277695 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.277696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.278301 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.278302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.278814 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.278815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.279109 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.279110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.279148 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.279562 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:27.279566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:27.281274 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260926 13:04:27.281502 11550 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:27.277124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41419 (local address 127.2.86.62:39575)
0926 13:04:27.277243 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:27.277245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:27.277273 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:04:27.277357 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:27.277358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:27.277369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:27.277450 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:27.277755 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.278238 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.278875 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.279051 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.279098 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.281405 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:41419: BlockingRecv error: recv got EOF from 127.2.86.1:41419 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:27.417043 11551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:27.412352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:40183)
0926 13:04:27.412531 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:27.412544 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:27.412622 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:27.412820 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:27.412821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:27.413022 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:27.413070 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.413072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.413732 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.413733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.414265 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.414266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.414648 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.414649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.414713 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.415172 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:27.415176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:27.416979 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
W20260926 13:04:27.417239 11550 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:27.412452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:40183 (local address 127.2.86.62:39575)
0926 13:04:27.412523 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:27.412526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:27.412604 (+ 78us) server_negotiation.cc:407] Connection header received
0926 13:04:27.412704 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:27.412705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:27.412734 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:27.412815 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:27.413149 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.413648 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.414347 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.414561 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.414647 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.417179 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:40183: BlockingRecv error: recv got EOF from 127.2.86.2:40183 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:27 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:27.538390 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:27.533606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:34695)
0926 13:04:27.533753 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:27.533764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:27.533827 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:27.534071 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:27.534071 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:27.534242 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:27.534288 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.534289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.535006 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.535007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.535732 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:27.535734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.536108 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.536109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:27.536147 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.536578 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:27.536583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:27.538329 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
W20260926 13:04:27.538484 11550 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:27.533898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34695 (local address 127.2.86.62:39575)
0926 13:04:27.533956 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:27.533958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:27.533964 (+ 6us) server_negotiation.cc:407] Connection header received
0926 13:04:27.533982 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:27.533983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:27.533995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:27.534061 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:27.534364 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.534857 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.535805 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:27.536010 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:27.536079 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:27.538417 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:34695: BlockingRecv error: recv got EOF from 127.2.86.3:34695 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:28.286187 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:28.281791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:41777)
0926 13:04:28.281922 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:28.281932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:28.281988 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:28.282122 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:28.282122 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:28.282274 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:28.282322 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.282324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.282981 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.282982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.283510 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.283512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.283836 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.283836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.283874 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.284319 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:28.284323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:28.286119 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
W20260926 13:04:28.286307 11554 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:28.281858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41777 (local address 127.2.86.62:39575)
0926 13:04:28.281950 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:28.281952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:28.281973 (+ 21us) server_negotiation.cc:407] Connection header received
0926 13:04:28.282043 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:28.282043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:28.282053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:28.282110 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:28.282377 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.282916 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.283588 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.283775 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.283823 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.286207 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:41777: BlockingRecv error: recv got EOF from 127.2.86.1:41777 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":61,"threads_started":1}
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:28.422101 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:28.417495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:50987)
0926 13:04:28.417632 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:28.417643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:28.417699 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:28.417828 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:28.417829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:28.417986 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:28.418033 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.418035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.418649 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.418650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.419176 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.419178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.419481 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.419482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.419522 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.420058 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:28.420065 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:28.422035 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
W20260926 13:04:28.422219 11554 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:28.417581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:50987 (local address 127.2.86.62:39575)
0926 13:04:28.417645 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:28.417646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:28.417701 (+ 55us) server_negotiation.cc:407] Connection header received
0926 13:04:28.417740 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:28.417740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:28.417751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:28.417803 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:28.418091 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.418582 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.419241 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.419418 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.419467 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.422122 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:50987: BlockingRecv error: recv got EOF from 127.2.86.2:50987 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:28 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:28.543159 11556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:28.538878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:57639)
0926 13:04:28.539025 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:28.539035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:28.539091 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:28.539208 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:28.539209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:28.539362 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:28.539411 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.539412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.540018 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.540018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.540543 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:28.540544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.540850 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.540850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:28.540915 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.541353 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:28.541357 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:28.543092 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
W20260926 13:04:28.543226 11554 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:28.538940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:57639 (local address 127.2.86.62:39575)
0926 13:04:28.539015 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:28.539017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:28.539093 (+ 76us) server_negotiation.cc:407] Connection header received
0926 13:04:28.539129 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:28.539130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:28.539141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:28.539191 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:28.539470 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.539947 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.540619 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:28.540790 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:28.540839 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:28.543177 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:57639: BlockingRecv error: recv got EOF from 127.2.86.3:57639 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:29.290850 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:29.286630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:50409)
0926 13:04:29.286759 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:29.286768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:29.286813 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:29.286958 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:29.286959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:29.287104 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:29.287151 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.287153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.287760 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.287761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.288282 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.288283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.288594 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.288595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.288632 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.289085 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:29.289089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:29.290793 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
W20260926 13:04:29.290975 11558 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:29.286688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:50409 (local address 127.2.86.62:39575)
0926 13:04:29.286809 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:29.286811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:29.286818 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:29.286875 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:29.286876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:29.286885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:29.286933 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:29.287212 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.287694 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.288349 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.288536 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.288584 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.290882 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:50409: BlockingRecv error: recv got EOF from 127.2.86.1:50409 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:29.433161 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:29.422516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:53313)
0926 13:04:29.422687 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:29.422704 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:29.422828 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:29.423003 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:29.423004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:29.423214 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:29.423285 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.423287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.424042 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.424043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.424557 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.424558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.424945 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.424946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.425003 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.431197 (+ 6194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:29.431202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:29.433061 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
W20260926 13:04:29.433223 11558 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:29.422612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:53313 (local address 127.2.86.62:39575)
0926 13:04:29.422666 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:29.422668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:29.422788 (+ 120us) server_negotiation.cc:407] Connection header received
0926 13:04:29.422905 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:29.422907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:29.422920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:29.422976 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:29.423359 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.423976 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.424630 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.424812 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.424878 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.433143 (+ 8265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:53313: BlockingRecv error: recv got EOF from 127.2.86.2:53313 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:29 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:29.548092 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:29.543627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:38133)
0926 13:04:29.543830 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:29.543843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:29.543931 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:29.544062 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:29.544063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:29.544238 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:29.544287 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.544289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.544977 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.544978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.545504 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:29.545506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.545865 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.545866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:29.545921 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.546332 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:29.546337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:29.547982 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":111,"threads_started":1}
W20260926 13:04:29.548141 11558 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:29.543739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38133 (local address 127.2.86.62:39575)
0926 13:04:29.543834 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:29.543836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:29.543928 (+ 92us) server_negotiation.cc:407] Connection header received
0926 13:04:29.543989 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:29.543990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:29.544004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:29.544058 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:29.544371 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.544860 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.545589 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:29.545791 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:29.545867 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:29.548077 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:38133: BlockingRecv error: recv got EOF from 127.2.86.3:38133 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:30.295691 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:30.291317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:35757)
0926 13:04:30.291458 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:30.291468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:30.291517 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:30.291644 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:30.291645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:30.291794 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:30.291841 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.291842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.292457 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.292457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.293062 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.293064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.293392 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.293393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.293431 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.293932 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:30.293937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:30.295633 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
W20260926 13:04:30.295818 11562 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:30.291381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:35757 (local address 127.2.86.62:39575)
0926 13:04:30.291506 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:30.291508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:30.291517 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:30.291565 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:30.291565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:30.291575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:30.291633 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:30.291897 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.292389 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.293132 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.293330 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.293381 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.295748 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:35757: BlockingRecv error: recv got EOF from 127.2.86.1:35757 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:30.438355 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:30.433836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:59753)
0926 13:04:30.433976 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:30.433989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:30.434055 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:30.434222 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:30.434223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:30.434398 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:30.434445 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.434446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.435102 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.435103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.435628 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.435629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.435995 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.435996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.436058 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.436512 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:30.436516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:30.438262 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
W20260926 13:04:30.438395 11562 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:30.433942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:59753 (local address 127.2.86.62:39575)
0926 13:04:30.434003 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:30.434005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:30.434035 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:04:30.434132 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:30.434133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:30.434146 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:30.434218 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:30.434520 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.435018 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.435710 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.435923 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.435998 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.438336 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:59753: BlockingRecv error: recv got EOF from 127.2.86.2:59753 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:30 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:30.552636 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:30.548534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:33139)
0926 13:04:30.548683 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:30.548693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:30.548744 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:30.548849 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:30.548849 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:30.549028 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:30.549074 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.549076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.549686 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.549687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.550191 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:30.550192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.550498 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.550498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:30.550536 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.550953 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:30.550957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:30.552580 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
W20260926 13:04:30.552766 11562 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:30.548581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:33139 (local address 127.2.86.62:39575)
0926 13:04:30.548650 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:30.548653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:30.548747 (+ 94us) server_negotiation.cc:407] Connection header received
0926 13:04:30.548785 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:30.548786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:30.548797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:30.548843 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:30.549133 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.549622 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.550257 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:30.550429 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:30.550492 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:30.552686 (+ 2194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:33139: BlockingRecv error: recv got EOF from 127.2.86.3:33139 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:31.300626 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:31.296339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:37203)
0926 13:04:31.296493 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:31.296511 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:31.296565 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:31.296691 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:31.296692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:31.296857 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:31.296925 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.296927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.297552 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.297552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.298060 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.298061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.298382 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.298382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.298426 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.298848 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:31.298852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:31.300570 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
W20260926 13:04:31.300770 11566 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:31.296391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:37203 (local address 127.2.86.62:39575)
0926 13:04:31.296506 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:31.296508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:31.296559 (+ 51us) server_negotiation.cc:407] Connection header received
0926 13:04:31.296605 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:31.296606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:31.296615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:31.296687 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:31.296974 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.297484 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.298129 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.298318 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.298370 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.300677 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:37203: BlockingRecv error: recv got EOF from 127.2.86.1:37203 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:31.443020 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:31.438822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:51491)
0926 13:04:31.438966 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:31.438988 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:31.439052 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:31.439174 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:31.439175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:31.439339 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:31.439387 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.439388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.439996 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.439997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.440520 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.440522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.440820 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.440821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.440857 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.441323 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:31.441327 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:31.442963 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260926 13:04:31.443159 11566 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:31.438887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51491 (local address 127.2.86.62:39575)
0926 13:04:31.438933 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:31.438935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:31.439055 (+ 120us) server_negotiation.cc:407] Connection header received
0926 13:04:31.439095 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:31.439096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:31.439106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:31.439154 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:31.439448 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.439926 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.440587 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.440761 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.440809 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.443071 (+ 2262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:51491: BlockingRecv error: recv got EOF from 127.2.86.2:51491 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:31 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:31.557763 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:31.553142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:54331)
0926 13:04:31.553304 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:31.553317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:31.553397 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:31.553544 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:31.553545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:31.553712 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:31.553757 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.553759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.554429 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.554430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.554972 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:31.554973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.555358 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.555359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:31.555422 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.555842 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:31.555846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:31.557672 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
W20260926 13:04:31.557808 11566 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:31.553224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:54331 (local address 127.2.86.62:39575)
0926 13:04:31.553293 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:31.553296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:31.553378 (+ 82us) server_negotiation.cc:407] Connection header received
0926 13:04:31.553471 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:31.553473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:31.553487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:31.553539 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:31.553842 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.554337 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.555055 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:31.555273 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:31.555356 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:31.557744 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:54331: BlockingRecv error: recv got EOF from 127.2.86.3:54331 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:32.305677 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:32.301096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:54963)
0926 13:04:32.301235 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:32.301245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:32.301292 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:32.301425 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:32.301426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:32.301577 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:32.301626 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.301627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.302233 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.302233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.302749 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.302751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.303061 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.303061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.303099 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.303577 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:32.303582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:32.305586 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260926 13:04:32.305771 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:32.301158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:54963 (local address 127.2.86.62:39575)
0926 13:04:32.301302 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:32.301304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:32.301312 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:32.301333 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:32.301333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:32.301343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:32.301404 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:32.301689 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.302170 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.302820 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.303001 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.303048 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.305687 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:54963: BlockingRecv error: recv got EOF from 127.2.86.1:54963 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:32.447904 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:32.443468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:52555)
0926 13:04:32.443634 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:32.443646 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:32.443734 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:32.443930 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:32.443931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:32.444141 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:32.444205 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.444207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.444850 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.444851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.445471 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.445474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.445775 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.445775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.445813 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.446242 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:32.446246 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:32.447849 (+ 1603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
W20260926 13:04:32.448029 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:32.443553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:52555 (local address 127.2.86.62:39575)
0926 13:04:32.443621 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:32.443624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:32.443798 (+ 174us) server_negotiation.cc:407] Connection header received
0926 13:04:32.443827 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:32.443827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:32.443841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:32.443897 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:32.444283 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.444789 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.445542 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.445717 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.445766 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.447936 (+ 2170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:52555: BlockingRecv error: recv got EOF from 127.2.86.2:52555 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:32 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:32.563035 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:32.558243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:33911)
0926 13:04:32.558426 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:32.558438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:32.558540 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:32.558687 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:32.558687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:32.558857 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:32.558901 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.558903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.559546 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.559548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.560073 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:32.560074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.560463 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.560464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:32.560533 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.560971 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:32.560977 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:32.562940 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
W20260926 13:04:32.563068 11570 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:32.558334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:33911 (local address 127.2.86.62:39575)
0926 13:04:32.558410 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:32.558413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:32.558519 (+ 106us) server_negotiation.cc:407] Connection header received
0926 13:04:32.558605 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:32.558606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:32.558620 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:32.558678 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:32.558986 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.559467 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.560165 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:32.560353 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:32.560463 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:32.563010 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:33911: BlockingRecv error: recv got EOF from 127.2.86.3:33911 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:33.310405 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:33.306081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:55125)
0926 13:04:33.306220 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:33.306230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:33.306279 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:33.306448 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:33.306449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:33.306602 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:33.306650 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.306652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.307274 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.307274 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.307790 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.307792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.308114 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.308115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.308152 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.308577 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:33.308581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:33.310311 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
W20260926 13:04:33.310474 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:33.306193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:55125 (local address 127.2.86.62:39575)
0926 13:04:33.306324 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:33.306326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:33.306334 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:33.306353 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:33.306354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:33.306364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:33.306423 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:33.306714 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.307209 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.307859 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.308052 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.308100 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.310402 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:55125: BlockingRecv error: recv got EOF from 127.2.86.1:55125 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:33.453187 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:33.448433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:51419)
0926 13:04:33.448578 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:33.448589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:33.448646 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:33.448773 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:33.448774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:33.448995 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:33.449044 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.449046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.449789 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.449789 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.450516 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.450520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.450843 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.450844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.450895 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.451330 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:33.451334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:33.453129 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
W20260926 13:04:33.453289 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:33.448548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:51419 (local address 127.2.86.62:39575)
0926 13:04:33.448609 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:33.448611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:33.448637 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:04:33.448691 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:33.448691 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:33.448701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:33.448754 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:33.449111 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.449726 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.450598 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.450779 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.450829 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.453216 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:51419: BlockingRecv error: recv got EOF from 127.2.86.2:51419 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:33 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:33.567673 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:33.563525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:55425)
0926 13:04:33.563657 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:33.563668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:33.563719 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:33.563821 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:33.563821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:33.563995 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:33.564041 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.564042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.564629 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.564630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.565167 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:33.565168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.565456 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.565456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:33.565493 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.565900 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:33.565904 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:33.567581 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
W20260926 13:04:33.567728 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:33.563570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:55425 (local address 127.2.86.62:39575)
0926 13:04:33.563610 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:33.563612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:33.563722 (+ 110us) server_negotiation.cc:407] Connection header received
0926 13:04:33.563758 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:33.563759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:33.563771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:33.563815 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:33.564095 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.564566 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.565227 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:33.565397 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:33.565445 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:33.567661 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:55425: BlockingRecv error: recv got EOF from 127.2.86.3:55425 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:34.315241 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:34.310842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:34207)
0926 13:04:34.311006 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:34.311015 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:34.311084 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:34.311195 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:34.311195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:34.311346 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:34.311393 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.311394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.312008 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.312009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.312551 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.312552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.312857 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.312858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.312950 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.313430 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:34.313435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:34.315186 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
W20260926 13:04:34.315354 11578 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:34.310928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:34207 (local address 127.2.86.62:39575)
0926 13:04:34.311024 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:34.311026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:34.311086 (+ 60us) server_negotiation.cc:407] Connection header received
0926 13:04:34.311125 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:34.311126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:34.311135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:34.311191 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:34.311447 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.311944 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.312613 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.312800 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.312847 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.315272 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:34207: BlockingRecv error: recv got EOF from 127.2.86.1:34207 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":45,"threads_started":1}
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:34.462980 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:34.453859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:54411)
0926 13:04:34.453975 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:34.453999 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:34.454048 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:34.454187 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:34.454188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:34.454345 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:34.454390 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.454391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.454997 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.454998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.455503 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.455505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.455808 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.455808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.455846 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.461099 (+ 5253us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:34.461104 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:34.462920 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
W20260926 13:04:34.463107 11578 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:34.453956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:54411 (local address 127.2.86.62:39575)
0926 13:04:34.454025 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:34.454027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:34.454044 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:04:34.454105 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:34.454105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:34.454115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:34.454171 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:34.454456 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.454929 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.455571 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.455750 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.455797 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.463028 (+ 7231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:54411: BlockingRecv error: recv got EOF from 127.2.86.2:54411 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:34 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:34.572723 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:34.568141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:51559)
0926 13:04:34.568312 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:34.568328 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:34.568401 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:34.568557 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:34.568558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:34.568733 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:34.568776 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.568777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.569469 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.569470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.570009 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:34.570010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.570352 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.570352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:34.570413 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.570851 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:34.570856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:34.572614 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
W20260926 13:04:34.572789 11578 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:34.568201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:51559 (local address 127.2.86.62:39575)
0926 13:04:34.568269 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:34.568272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:34.568390 (+ 118us) server_negotiation.cc:407] Connection header received
0926 13:04:34.568458 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:34.568459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:34.568473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:34.568551 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:34.568890 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.569378 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.570089 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:34.570272 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:34.570352 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:34.572703 (+ 2351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:51559: BlockingRecv error: recv got EOF from 127.2.86.3:51559 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:35.320228 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:35.315730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:57197)
0926 13:04:35.315855 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:35.315866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:35.315929 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:35.316093 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:35.316093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:35.316261 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:35.316309 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.316310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.317013 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.317014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.317531 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.317532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.317863 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.317864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.317903 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.318419 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:35.318425 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:35.320172 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
W20260926 13:04:35.320384 11582 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:35.315775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57197 (local address 127.2.86.62:39575)
0926 13:04:35.315950 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:35.315952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:35.315962 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:35.315993 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:35.315993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:35.316006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:35.316078 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:35.316382 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.316947 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.317600 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.317789 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.317842 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.320283 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:57197: BlockingRecv error: recv got EOF from 127.2.86.1:57197 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":112,"threads_started":1}
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:35.467941 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:35.463444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:35651)
0926 13:04:35.463591 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:35.463602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:35.463665 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:35.463790 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:35.463790 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:35.463964 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:35.464012 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.464014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.464673 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.464673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.465264 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.465266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.465601 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.465602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.465640 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.466149 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:35.466154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:35.467868 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
W20260926 13:04:35.468205 11582 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:35.463495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35651 (local address 127.2.86.62:39575)
0926 13:04:35.463552 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:35.463555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:35.463648 (+ 93us) server_negotiation.cc:407] Connection header received
0926 13:04:35.463715 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:35.463716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:35.463728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:35.463769 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:35.464083 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.464585 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.465354 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.465539 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.465594 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.468118 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:35651: BlockingRecv error: recv got EOF from 127.2.86.2:35651 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:35 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:35.577793 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:35.573211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:38379)
0926 13:04:35.573354 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:35.573366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:35.573430 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:35.573551 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:35.573552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:35.573730 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:35.573775 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.573776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.574447 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.574449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.574974 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:35.574976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.575343 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.575344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:35.575410 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.575974 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:35.575979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:35.577738 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260926 13:04:35.577971 11582 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:35.573322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:38379 (local address 127.2.86.62:39575)
0926 13:04:35.573370 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:35.573372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:35.573413 (+ 41us) server_negotiation.cc:407] Connection header received
0926 13:04:35.573484 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:35.573485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:35.573499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:35.573547 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:35.573879 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.574365 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.575058 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:35.575253 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:35.575336 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:35.577895 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:38379: BlockingRecv error: recv got EOF from 127.2.86.3:38379 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:36.325165 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:36.320721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:42695)
0926 13:04:36.320908 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:36.320928 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:36.320977 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:36.321102 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:36.321102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:36.321256 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:36.321304 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.321306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.321930 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.321931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.322434 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.322435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.322752 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.322752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.322789 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.323333 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:36.323337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:36.325097 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":113,"threads_started":1}
W20260926 13:04:36.325299 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:36.320791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:42695 (local address 127.2.86.62:39575)
0926 13:04:36.320964 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:36.320966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:36.320976 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:36.321034 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:36.321035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:36.321045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:36.321095 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:36.321370 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.321857 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.322497 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.322692 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.322740 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.325201 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:42695: BlockingRecv error: recv got EOF from 127.2.86.1:42695 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":128,"threads_started":1}
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 13:04:36.473806 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:36.468516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:46047 (local address 127.2.86.62:39575)
0926 13:04:36.468586 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:36.468589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:36.468772 (+ 183us) server_negotiation.cc:407] Connection header received
0926 13:04:36.468847 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:36.468848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:36.468863 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:36.468931 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:36.469299 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.470202 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.470932 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.471290 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.471367 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.473760 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:46047: BlockingRecv error: recv got EOF from 127.2.86.2:46047 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:04:36.473798 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:36.468444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:46047)
0926 13:04:36.468701 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:36.468713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:36.468776 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:36.468959 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:36.468960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:36.469178 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:36.469232 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.469235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.470283 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.470284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.470854 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.470856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.471368 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.471369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.471405 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.471869 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:36.471873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:36.473684 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":79,"threads_started":1}
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:36 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:36.582791 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:36.578236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:36939)
0926 13:04:36.578398 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:36.578408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:36.578479 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:36.578630 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:36.578630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:36.578787 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:36.578849 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.578850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.579492 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.579492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.580029 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:36.580030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.580317 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.580317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:36.580361 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.580913 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:36.580926 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:36.582718 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
W20260926 13:04:36.582870 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:36.578278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:36939 (local address 127.2.86.62:39575)
0926 13:04:36.578342 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:36.578345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:36.578522 (+ 177us) server_negotiation.cc:407] Connection header received
0926 13:04:36.578549 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:36.578549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:36.578563 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:36.578618 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:36.578941 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.579422 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.580092 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:36.580259 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:36.580306 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:36.582792 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:36939: BlockingRecv error: recv got EOF from 127.2.86.3:36939 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:37.330205 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:37.325608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:57761)
0926 13:04:37.325759 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:37.325770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:37.325828 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:37.325950 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:37.325950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:37.326106 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:37.326153 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.326154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.326770 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.326771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.327385 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.327387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.327703 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.327703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.327741 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.328284 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:37.328288 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:37.330139 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
W20260926 13:04:37.330355 11590 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:37.325694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57761 (local address 127.2.86.62:39575)
0926 13:04:37.325825 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:37.325827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:37.325838 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:04:37.325868 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:37.325868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:37.325887 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:37.325944 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:37.326213 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.326706 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.327450 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.327638 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.327692 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.330231 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:57761: BlockingRecv error: recv got EOF from 127.2.86.1:57761 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:37.478638 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:37.474267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:32999)
0926 13:04:37.474424 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:37.474441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:37.474499 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:37.474633 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:37.474634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:37.474792 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:37.474840 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.474841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.475442 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.475442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.475964 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.475965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.476258 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.476259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.476294 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.476737 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:37.476741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:37.478574 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260926 13:04:37.478735 11590 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:37.474329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:32999 (local address 127.2.86.62:39575)
0926 13:04:37.474376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:37.474378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:37.474501 (+ 123us) server_negotiation.cc:407] Connection header received
0926 13:04:37.474555 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:37.474556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:37.474567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:37.474608 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:37.474904 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.475375 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.476029 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.476200 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.476246 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.478668 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:32999: BlockingRecv error: recv got EOF from 127.2.86.2:32999 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:37 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:37.588003 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:37.583259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:56801)
0926 13:04:37.583403 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:37.583415 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:37.583483 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:37.583645 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:37.583646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:37.583915 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:37.583991 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.583993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.584711 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.584712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.585373 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:37.585375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.585708 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.585709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:37.585768 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.586225 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:37.586229 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:37.587938 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":117,"threads_started":1}
W20260926 13:04:37.588111 11590 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:37.583326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:56801 (local address 127.2.86.62:39575)
0926 13:04:37.583367 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:37.583369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:37.583482 (+ 113us) server_negotiation.cc:407] Connection header received
0926 13:04:37.583523 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:37.583524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:37.583536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:37.583586 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:37.584061 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.584639 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.585450 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:37.585626 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:37.585680 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:37.588032 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:56801: BlockingRecv error: recv got EOF from 127.2.86.3:56801 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:38.335330 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:38.330652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:49547)
0926 13:04:38.330775 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:38.330788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:38.330849 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:38.330984 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:38.330985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:38.331142 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:38.331189 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.331191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.331990 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.331991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.332518 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.332520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.332908 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.332909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.332974 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.333395 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:38.333399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:38.335265 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
W20260926 13:04:38.335486 11594 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:38.330713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:49547 (local address 127.2.86.62:39575)
0926 13:04:38.330841 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:38.330843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:38.330851 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:38.330901 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:38.330901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:38.330912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:38.330969 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:38.331276 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.331880 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.332586 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.332793 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.332860 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.335376 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:49547: BlockingRecv error: recv got EOF from 127.2.86.1:49547 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:38.483332 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:38.479152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:43245)
0926 13:04:38.479271 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:38.479282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:38.479339 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:38.479465 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:38.479466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:38.479621 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:38.479668 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.479669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.480263 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.480264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.480770 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.480771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.481115 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.481116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.481158 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.481568 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:38.481572 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:38.483249 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
W20260926 13:04:38.483474 11594 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:38.479229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:43245 (local address 127.2.86.62:39575)
0926 13:04:38.479302 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:38.479305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:38.479324 (+ 19us) server_negotiation.cc:407] Connection header received
0926 13:04:38.479400 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:38.479400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:38.479410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:38.479461 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:38.479723 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.480197 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.480847 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.481048 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.481096 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.483377 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:43245: BlockingRecv error: recv got EOF from 127.2.86.2:43245 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:38 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:38.592731 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:38.588479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:43599)
0926 13:04:38.588636 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:38.588647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:38.588704 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:38.588832 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:38.588833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:38.589019 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:38.589064 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.589066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.589667 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.589667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.590176 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:38.590177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.590472 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.590473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:38.590510 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.590968 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:38.590972 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:38.592668 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
W20260926 13:04:38.592906 11594 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:38.588548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:43599 (local address 127.2.86.62:39575)
0926 13:04:38.588629 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:38.588631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:38.588693 (+ 62us) server_negotiation.cc:407] Connection header received
0926 13:04:38.588748 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:38.588749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:38.588761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:38.588812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:38.589124 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.589605 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.590241 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:38.590413 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:38.590461 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:38.592752 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:43599: BlockingRecv error: recv got EOF from 127.2.86.3:43599 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:39.340682 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:39.335885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:43999)
0926 13:04:39.336024 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:39.336043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:39.336090 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:39.336244 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:39.336245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:39.336406 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:39.336456 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.336458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.337109 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.337110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.337653 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.337654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.337965 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.337965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.338004 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.338524 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:39.338528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:39.340611 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260926 13:04:39.340816 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:39.335996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:43999 (local address 127.2.86.62:39575)
0926 13:04:39.336125 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:39.336127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:39.336135 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:39.336154 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:39.336155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:39.336164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:39.336226 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:39.336520 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.337038 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.337723 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.337904 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.337951 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.340706 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:43999: BlockingRecv error: recv got EOF from 127.2.86.1:43999 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:39.495175 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:39.483969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:37829)
0926 13:04:39.484150 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:39.484163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:39.484251 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:39.484408 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:39.484426 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:39.484665 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:39.484721 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.484723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.485435 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.485436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.485961 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.485963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.486329 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.486330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.486386 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.493324 (+ 6938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:39.493330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:39.495078 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
W20260926 13:04:39.495224 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:39.484039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:37829 (local address 127.2.86.62:39575)
0926 13:04:39.484131 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:39.484133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:39.484232 (+ 99us) server_negotiation.cc:407] Connection header received
0926 13:04:39.484338 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:39.484339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:39.484350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:39.484401 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:39.484797 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.485352 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.486043 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.486241 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.486328 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.495161 (+ 8833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:37829: BlockingRecv error: recv got EOF from 127.2.86.2:37829 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:39 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.3@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:39.597662 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:39.593181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.3:56097)
0926 13:04:39.593356 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:39.593365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:39.593417 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:39.593524 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:39.593524 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:39.593678 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:39.593727 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.593728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.594331 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.594332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.594868 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:39.594870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.595183 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.595183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:39.595224 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.595691 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:39.595696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:39.597569 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
W20260926 13:04:39.597719 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:39.593255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:56097 (local address 127.2.86.62:39575)
0926 13:04:39.593307 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:39.593309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:39.593403 (+ 94us) server_negotiation.cc:407] Connection header received
0926 13:04:39.593459 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:39.593460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:39.593470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:39.593519 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:39.593778 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.594254 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.594940 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:39.595117 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:39.595168 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:39.597655 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.3:56097: BlockingRecv error: recv got EOF from 127.2.86.3:56097 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 26 13:04:40 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:40 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.1@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:40.346099 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.341368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.1:51203)
0926 13:04:40.341537 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:40.341548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:40.341618 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:40.341806 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:40.341807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:40.341969 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:40.342016 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.342017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.342702 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.342703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.343239 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.343241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.343572 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.343572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.343610 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.344114 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:40.344118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:40.346035 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260926 13:04:40.346248 11602 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:40.341483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:51203 (local address 127.2.86.62:39575)
0926 13:04:40.341643 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:40.341645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:40.341654 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:04:40.341678 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:40.341679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:40.341689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:40.341761 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:40.342104 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.342623 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.343317 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.343510 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.343563 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.346156 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.1:51203: BlockingRecv error: recv got EOF from 127.2.86.1:51203 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
Sep 26 13:04:40 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 13:04:40 dist-test-slave-j4lq krb5kdc[10544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.86.2@KRBTEST.COM for oryx/127.2.86.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 13:04:40.500425 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.495686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39575 (local address 127.2.86.2:47385)
0926 13:04:40.495885 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:40.495898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:40.495961 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:40.496134 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:40.496134 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:40.496311 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:40.496356 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.496358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.497187 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.497188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.497713 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.497715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.498064 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.498065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.498104 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.498592 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:40.498602 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:40.500350 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.86.62:39575: Server oryx/127.2.86.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":68,"threads_started":1}
W20260926 13:04:40.500536 11602 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:40.495742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:47385 (local address 127.2.86.62:39575)
0926 13:04:40.495792 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:40.495795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:40.495943 (+ 148us) server_negotiation.cc:407] Connection header received
0926 13:04:40.496050 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:40.496051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:40.496064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:40.496129 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:40.496420 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.497111 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.497781 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.497988 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.498056 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.500458 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.86.2:47385: BlockingRecv error: recv got EOF from 127.2.86.2:47385 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 13:04:40.527597 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11128
I20260926 13:04:40.532172 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11279
I20260926 13:04:40.536157 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11412
I20260926 13:04:40.540334 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11057
2026-09-26T13:04:40Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17415 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T13:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:40Z Disabled control of system clock
I20260926 13:04:40.567332 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39871
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:39871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 13:04:40.645486 11607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:40.645678 11607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:40.645697 11607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:40.646723 11607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:40.646754 11607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:40.646766 11607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:40.646778 11607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:40.646790 11607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:40.648049 11607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:39871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:39871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:40.648224 11607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:40.648445 11607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:04:40.653095 11607 server_base.cc:1034] running on GCE node
W20260926 13:04:40.653065 11612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:40.653059 11613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:40.653075 11615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:40.653543 11607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:40.653914 11607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:40.655090 11607 hybrid_clock.cc:648] HybridClock initialized: now 1790427880655068 us; error 34 us; skew 500 ppm
I20260926 13:04:40.656229 11607 webserver.cc:460] Webserver started at http://127.2.86.62:37631/ using document root <none> and password file <none>
I20260926 13:04:40.656484 11607 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:40.656560 11607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:40.656693 11607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:40.657873 11607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "9a0c7b798aad4abebca069f1857b7622"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "cde78ec641e1abcedf8a7751f2ef4170"
server_key_iv: "983df3d55df19f17f2e40d062ae44a52"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.658253 11607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "9a0c7b798aad4abebca069f1857b7622"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "cde78ec641e1abcedf8a7751f2ef4170"
server_key_iv: "983df3d55df19f17f2e40d062ae44a52"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.659585 11607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:40.660364 11621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.660537 11607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:40.660633 11607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "9a0c7b798aad4abebca069f1857b7622"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "cde78ec641e1abcedf8a7751f2ef4170"
server_key_iv: "983df3d55df19f17f2e40d062ae44a52"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.660717 11607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:40.669973 11607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:40.670262 11607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:40.670416 11607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:40.674261 11607 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:39871
I20260926 13:04:40.674297 11673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:39871 every 8 connection(s)
I20260926 13:04:40.674649 11607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:40.675300 11674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:40.677883 11674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622: Bootstrap starting.
I20260926 13:04:40.678408 11674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:40.678630 11674 log.cc:824] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:40.679236 11674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622: No bootstrap required, opened a new log
I20260926 13:04:40.681062 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11607
I20260926 13:04:40.681011 11674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } }
I20260926 13:04:40.681161 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:40.681135 11674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:40.681190 11674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0c7b798aad4abebca069f1857b7622, State: Initialized, Role: FOLLOWER
I20260926 13:04:40.681337 11674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } }
I20260926 13:04:40.681398 11674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:40.681396 2392 external_mini_cluster.cc:1428] Setting key e7cda4ec6bcb81e4f5a05d7bd8c56b5a
I20260926 13:04:40.681440 11674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:40.681490 11674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:40.682200 11674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } }
I20260926 13:04:40.682283 11674 leader_election.cc:302] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a0c7b798aad4abebca069f1857b7622; no voters:
I20260926 13:04:40.682466 11674 leader_election.cc:288] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:40.682518 11679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:40.682664 11679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [term 1 LEADER]: Becoming Leader. State: Replica: 9a0c7b798aad4abebca069f1857b7622, State: Running, Role: LEADER
I20260926 13:04:40.682765 11679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } }
I20260926 13:04:40.682777 11674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:40.683218 11680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a0c7b798aad4abebca069f1857b7622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } } }
I20260926 13:04:40.683216 11681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a0c7b798aad4abebca069f1857b7622. Latest consensus state: current_term: 1 leader_uuid: "9a0c7b798aad4abebca069f1857b7622" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0c7b798aad4abebca069f1857b7622" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 39871 } } }
I20260926 13:04:40.683336 11680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:40.683342 11681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:40.683571 11683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:40.684082 11683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:40.685747 11683 catalog_manager.cc:1347] Generated new cluster ID: eab41a7a87064834aebcef75544c6322
I20260926 13:04:40.685806 11683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:40.687754 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.681875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.2.86.62:39871)
0926 13:04:40.682091 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:40.682097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:40.682121 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:04:40.682212 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:40.682218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:40.682293 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:40.682444 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:40.682735 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.684678 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.685700 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.686263 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.686379 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.687094 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:40.687098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:40.687098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:40.687284 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:40.687329 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:40.687341 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:40.687404 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:40.687496 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 13:04:40.687563 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260926 13:04:40.705644 11683 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:40.706418 11683 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:40.716336 11683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a0c7b798aad4abebca069f1857b7622: Generated new TSK 0
I20260926 13:04:40.716521 11683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:40.723393 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--builtin_ntp_servers=127.2.86.20:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 13:04:40.804557 11698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:40.804800 11698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:40.804847 11698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:40.806031 11698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:40.806097 11698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:40.806139 11698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:40.807468 11698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:40.807720 11698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:40.808015 11698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:40.812748 11706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:40.812758 11704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:40.813021 11698 server_base.cc:1034] running on GCE node
W20260926 13:04:40.812824 11703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:40.813355 11698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:40.813625 11698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:40.814760 11698 hybrid_clock.cc:648] HybridClock initialized: now 1790427880814741 us; error 37 us; skew 500 ppm
I20260926 13:04:40.816057 11698 webserver.cc:460] Webserver started at http://127.2.86.1:40705/ using document root <none> and password file <none>
I20260926 13:04:40.816309 11698 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:40.816357 11698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:40.816522 11698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:40.817658 11698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "8f97ae27e13d46ebbfe31e6d639f6983"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "7398ad8e4f3a289f2ea507d933582dcf"
server_key_iv: "c7dcfc4f009765dca40df3c2836f7f44"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.818013 11698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "8f97ae27e13d46ebbfe31e6d639f6983"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "7398ad8e4f3a289f2ea507d933582dcf"
server_key_iv: "c7dcfc4f009765dca40df3c2836f7f44"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.819272 11698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:40.819967 11712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.820152 11698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:40.820215 11698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "8f97ae27e13d46ebbfe31e6d639f6983"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "7398ad8e4f3a289f2ea507d933582dcf"
server_key_iv: "c7dcfc4f009765dca40df3c2836f7f44"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.820315 11698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:40.828176 11698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:40.828455 11698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:40.828601 11698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:40.828819 11698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:40.829155 11698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:40.829191 11698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.829211 11698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:40.829224 11698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.835346 11698 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:38741
I20260926 13:04:40.835387 11825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:38741 every 8 connection(s)
I20260926 13:04:40.835681 11698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:40.838201 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11698
I20260926 13:04:40.838300 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:40.838476 2392 external_mini_cluster.cc:1428] Setting key 59b287a4651002b5048f2df3197207e5
I20260926 13:04:40.839365 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--builtin_ntp_servers=127.2.86.20:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 13:04:40.841524 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.836355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:39675 (local address 127.2.86.62:39871)
0926 13:04:40.836406 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:40.836409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:40.836655 (+ 246us) server_negotiation.cc:407] Connection header received
0926 13:04:40.837286 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:40.837288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:40.837302 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:40.837379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:40.838039 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.838611 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.840069 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.840258 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.840328 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.841253 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:40.841254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:40.841255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:40.841260 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:40.841277 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:40.841281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:40.841362 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:40.841399 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 13:04:40.841426 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:40.841642 11828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.836354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39871 (local address 127.2.86.1:39675)
0926 13:04:40.836534 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:40.836565 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:40.837114 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:40.837462 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:40.837465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:40.837474 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:40.837950 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.837953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.838718 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.838719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.839983 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.839986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.840339 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.840340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.840500 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.841122 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:40.841123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:40.841148 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:40.841169 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:40.841344 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:40.841346 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:40.841348 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:40.841399 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 13:04:40.841467 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260926 13:04:40.842021 11826 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39871
I20260926 13:04:40.842103 11826 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:40.842281 11826 heartbeater.cc:511] Master 127.2.86.62:39871 requested a full tablet report, sending...
I20260926 13:04:40.842775 11638 ts_manager.cc:194] Registered new tserver with Master: 8f97ae27e13d46ebbfe31e6d639f6983 (127.2.86.1:38741)
I20260926 13:04:40.843390 11638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.1:39675
W20260926 13:04:40.919766 11829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:40.919968 11829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:40.919989 11829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:40.921248 11829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:40.921289 11829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:40.921314 11829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:40.922610 11829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:40.922892 11829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:40.923192 11829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:40.928014 11835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:40.927985 11834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:40.927992 11837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:40.928227 11829 server_base.cc:1034] running on GCE node
I20260926 13:04:40.928434 11829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:40.928634 11829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:40.929791 11829 hybrid_clock.cc:648] HybridClock initialized: now 1790427880929782 us; error 27 us; skew 500 ppm
I20260926 13:04:40.931068 11829 webserver.cc:460] Webserver started at http://127.2.86.2:41245/ using document root <none> and password file <none>
I20260926 13:04:40.931303 11829 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:40.931349 11829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:40.931515 11829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:40.932533 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "3b840815eaaf4848984d78dea9b3eeea"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "965467a9673a2fc918a871b8b1c29d77"
server_key_iv: "b9bcc4de80294a831fbcdc647fabe26c"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.932930 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "3b840815eaaf4848984d78dea9b3eeea"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "965467a9673a2fc918a871b8b1c29d77"
server_key_iv: "b9bcc4de80294a831fbcdc647fabe26c"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.934221 11829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:40.934964 11843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.935144 11829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:40.935245 11829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "3b840815eaaf4848984d78dea9b3eeea"
format_stamp: "Formatted at 2026-09-26 13:04:40 on dist-test-slave-j4lq"
server_key: "965467a9673a2fc918a871b8b1c29d77"
server_key_iv: "b9bcc4de80294a831fbcdc647fabe26c"
server_key_version: "encryptionkey@0"
I20260926 13:04:40.935329 11829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:40.957820 11829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:40.958166 11829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:40.958329 11829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:40.958575 11829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:40.958949 11829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:40.959004 11829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.959040 11829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:40.959069 11829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:40.965162 11829 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:33307
I20260926 13:04:40.965207 11956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:33307 every 8 connection(s)
I20260926 13:04:40.965523 11829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:40.970362 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.966244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:45969 (local address 127.2.86.62:39871)
0926 13:04:40.966308 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:40.966310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:40.966536 (+ 226us) server_negotiation.cc:407] Connection header received
0926 13:04:40.966995 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:40.966996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:40.967005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:40.967054 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:40.967679 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.968153 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.969175 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.969357 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.969408 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.970138 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:40.970140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:40.970140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:40.970144 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:40.970157 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:40.970161 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:40.970203 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:40.970285 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 13:04:40.970307 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:04:40.970542 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:40.966229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39871 (local address 127.2.86.2:45969)
0926 13:04:40.966416 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:40.966450 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:40.966912 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:40.967113 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:40.967116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:40.967124 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:40.967614 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.967619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.968233 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.968234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.969105 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:40.969107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:40.969430 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:40.969431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:40.969533 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:40.970011 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:40.970011 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:40.970039 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:40.970059 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:40.970215 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:40.970218 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:40.970221 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:40.970271 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:04:40.970351 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260926 13:04:40.970990 11957 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39871
I20260926 13:04:40.971076 11957 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:40.971212 11957 heartbeater.cc:511] Master 127.2.86.62:39871 requested a full tablet report, sending...
I20260926 13:04:40.971524 11638 ts_manager.cc:194] Registered new tserver with Master: 3b840815eaaf4848984d78dea9b3eeea (127.2.86.2:33307)
I20260926 13:04:40.971875 11638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.2:45969
I20260926 13:04:40.974571 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11829
I20260926 13:04:40.974644 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:40.974787 2392 external_mini_cluster.cc:1428] Setting key bc7e4d834d1005e332825b929be8b75d
I20260926 13:04:40.975592 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--builtin_ntp_servers=127.2.86.20:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 13:04:41.053757 11960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:41.053960 11960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:41.053992 11960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:41.055125 11960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:41.055166 11960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:41.055189 11960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:41.056481 11960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:39871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:41.056712 11960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:41.057065 11960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:41.062096 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:41.062076 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.062182 11960 server_base.cc:1034] running on GCE node
W20260926 13:04:41.062096 11965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.062568 11960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:41.062852 11960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:41.064009 11960 hybrid_clock.cc:648] HybridClock initialized: now 1790427881064001 us; error 40 us; skew 500 ppm
I20260926 13:04:41.065367 11960 webserver.cc:460] Webserver started at http://127.2.86.3:35459/ using document root <none> and password file <none>
I20260926 13:04:41.065614 11960 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:41.065683 11960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:41.065825 11960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:41.067104 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "358179312ba849c884d0f50ddc0d3401"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "7d0d0b75578fb4ad372e117db1c4cbe4"
server_key_iv: "ba78f2fcf474f1fd9f314684bda6b3a1"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.067562 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "358179312ba849c884d0f50ddc0d3401"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "7d0d0b75578fb4ad372e117db1c4cbe4"
server_key_iv: "ba78f2fcf474f1fd9f314684bda6b3a1"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.068845 11960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:04:41.069665 11974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.069929 11960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:41.070001 11960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "358179312ba849c884d0f50ddc0d3401"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "7d0d0b75578fb4ad372e117db1c4cbe4"
server_key_iv: "ba78f2fcf474f1fd9f314684bda6b3a1"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.070051 11960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:41.077389 11960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:41.077608 11960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:41.077706 11960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:41.077854 11960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:41.078105 11960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:41.078135 11960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.078155 11960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:41.078167 11960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.083967 11960 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:43873
I20260926 13:04:41.084023 12087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:43873 every 8 connection(s)
I20260926 13:04:41.084357 11960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:41.089181 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.085113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:41183 (local address 127.2.86.62:39871)
0926 13:04:41.085186 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.085188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.085406 (+ 218us) server_negotiation.cc:407] Connection header received
0926 13:04:41.085846 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.085847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.085857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.085907 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.086571 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:41.087035 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:41.087928 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:41.088096 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:41.088141 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:41.088930 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:41.088932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:41.088932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:41.088937 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:41.088953 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:41.088957 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:41.089014 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:41.089083 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 13:04:41.089108 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:04:41.089401 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.085090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:39871 (local address 127.2.86.3:41183)
0926 13:04:41.085280 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:41.085313 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:41.085761 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:41.085956 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:41.085959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:41.085965 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:41.086501 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:41.086506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:41.087105 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:41.087106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:41.087855 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:41.087857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:41.088159 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:41.088160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:41.088256 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:41.088735 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:41.088735 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:41.088760 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:41.088780 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:41.089017 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:41.089021 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:41.089023 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:41.089073 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:04:41.089146 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260926 13:04:41.089937 12088 heartbeater.cc:347] Connected to a master server at 127.2.86.62:39871
I20260926 13:04:41.090040 12088 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:41.090184 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 11960
I20260926 13:04:41.090209 12088 heartbeater.cc:511] Master 127.2.86.62:39871 requested a full tablet report, sending...
I20260926 13:04:41.090257 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:41.090467 2392 external_mini_cluster.cc:1428] Setting key 5727215f7da59e871d043b579beee1ce
I20260926 13:04:41.090600 11638 ts_manager.cc:194] Registered new tserver with Master: 358179312ba849c884d0f50ddc0d3401 (127.2.86.3:43873)
I20260926 13:04:41.091059 11638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.3:41183
I20260926 13:04:41.091328 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:04:41.093012 11678 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:41.092544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.2.86.62:39871)
0926 13:04:41.092597 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.092599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.092696 (+ 97us) server_negotiation.cc:407] Connection header received
0926 13:04:41.092751 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.092752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.092760 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.092801 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.092946 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58440: BlockingRecv error: recv got EOF from 127.0.0.1:58440 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:04:41.093070 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11698
I20260926 13:04:41.098284 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11829
I20260926 13:04:41.102682 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11960
I20260926 13:04:41.106941 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 11607
2026-09-26T13:04:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (566 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T13:04:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:41Z Disabled control of system clock
I20260926 13:04:41.132946 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45051
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:45051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:04:41.211213 12099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:41.211406 12099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:41.211427 12099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:41.212478 12099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:41.212512 12099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:41.212535 12099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:41.212548 12099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:41.212559 12099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:41.213877 12099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:45051
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:41.214061 12099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:41.214291 12099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:41.218940 12105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:41.218956 12107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.219018 12099 server_base.cc:1034] running on GCE node
W20260926 13:04:41.218966 12104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.219405 12099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:41.219660 12099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:41.220824 12099 hybrid_clock.cc:648] HybridClock initialized: now 1790427881220811 us; error 39 us; skew 500 ppm
I20260926 13:04:41.222159 12099 webserver.cc:460] Webserver started at http://127.2.86.62:38647/ using document root <none> and password file <none>
I20260926 13:04:41.222419 12099 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:41.222465 12099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:41.222631 12099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:41.223711 12099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "3cc793bc51c1478fad23c093d9d50515"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "c3cf01c9797d274cc6eef7281e19cd42"
server_key_iv: "c41a72d3e66ba1e401468672e0d9b1ff"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.224061 12099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "3cc793bc51c1478fad23c093d9d50515"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "c3cf01c9797d274cc6eef7281e19cd42"
server_key_iv: "c41a72d3e66ba1e401468672e0d9b1ff"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.225455 12099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:41.226250 12113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.226464 12099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:04:41.226554 12099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "3cc793bc51c1478fad23c093d9d50515"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "c3cf01c9797d274cc6eef7281e19cd42"
server_key_iv: "c41a72d3e66ba1e401468672e0d9b1ff"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.226629 12099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:41.229931 12099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:41.230213 12099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:41.230351 12099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:41.234040 12099 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:45051
I20260926 13:04:41.234086 12165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:45051 every 8 connection(s)
I20260926 13:04:41.234385 12099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:41.235018 12166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:41.236500 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12099
I20260926 13:04:41.236594 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:41.236790 2392 external_mini_cluster.cc:1428] Setting key e9e52be353570d66ecc4dd023433e768
I20260926 13:04:41.238201 12166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515: Bootstrap starting.
I20260926 13:04:41.238906 12166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:41.239251 12166 log.cc:824] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:41.239542 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.237300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.2.86.62:45051)
0926 13:04:41.237567 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.237575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.237605 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:04:41.238150 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.238158 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.238209 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.238396 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.239097 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:41.239101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:41.239101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:41.239106 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:41.239159 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:41.239176 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:41.239259 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:41.239295 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:04:41.239356 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":57,"threads_started":1}
I20260926 13:04:41.240188 12166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515: No bootstrap required, opened a new log
I20260926 13:04:41.241636 12166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } }
I20260926 13:04:41.241760 12166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:41.241781 12166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc793bc51c1478fad23c093d9d50515, State: Initialized, Role: FOLLOWER
I20260926 13:04:41.241891 12166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } }
I20260926 13:04:41.241972 12166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:41.241994 12166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:41.242022 12166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:41.242967 12166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } }
I20260926 13:04:41.243053 12166 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc793bc51c1478fad23c093d9d50515; no voters:
I20260926 13:04:41.243165 12166 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:41.243237 12171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:41.243404 12171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [term 1 LEADER]: Becoming Leader. State: Replica: 3cc793bc51c1478fad23c093d9d50515, State: Running, Role: LEADER
I20260926 13:04:41.243429 12166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:41.243489 12171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } }
I20260926 13:04:41.243858 12172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cc793bc51c1478fad23c093d9d50515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } } }
I20260926 13:04:41.243891 12173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc793bc51c1478fad23c093d9d50515. Latest consensus state: current_term: 1 leader_uuid: "3cc793bc51c1478fad23c093d9d50515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc793bc51c1478fad23c093d9d50515" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45051 } } }
I20260926 13:04:41.244062 12172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:41.244074 12173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515 [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:41.244539 12181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:41.245028 12181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:41.246385 12181 catalog_manager.cc:1347] Generated new cluster ID: 06ae5347b0754d93a3e22216c77529ce
I20260926 13:04:41.246445 12181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:04:41.261767 12181 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:41.261993 12181 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:41.271126 12181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cc793bc51c1478fad23c093d9d50515: Generated new TSK 0
I20260926 13:04:41.271329 12181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:04:41.273658 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--builtin_ntp_servers=127.2.86.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:04:41.357795 12190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:41.357995 12190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:41.358017 12190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:41.359171 12190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:41.359213 12190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:41.359238 12190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:41.360561 12190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:41.360793 12190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:41.361116 12190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:41.366827 12195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:41.366849 12196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.367064 12190 server_base.cc:1034] running on GCE node
W20260926 13:04:41.366832 12198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.367357 12190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:41.367604 12190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:41.368769 12190 hybrid_clock.cc:648] HybridClock initialized: now 1790427881368758 us; error 31 us; skew 500 ppm
I20260926 13:04:41.370078 12190 webserver.cc:460] Webserver started at http://127.2.86.1:45949/ using document root <none> and password file <none>
I20260926 13:04:41.370329 12190 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:41.370406 12190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:41.370563 12190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:41.371573 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "7b1dea6ebd7244a1b44229c33a6b7f81"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "250179f991ab39f2d9c5fa20fd1c8236"
server_key_iv: "ab6ef14bf77734b44fd67918c57a5a9a"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.371940 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "7b1dea6ebd7244a1b44229c33a6b7f81"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "250179f991ab39f2d9c5fa20fd1c8236"
server_key_iv: "ab6ef14bf77734b44fd67918c57a5a9a"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.373210 12190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:41.374008 12204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.374233 12190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:41.374317 12190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "7b1dea6ebd7244a1b44229c33a6b7f81"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "250179f991ab39f2d9c5fa20fd1c8236"
server_key_iv: "ab6ef14bf77734b44fd67918c57a5a9a"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.374384 12190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:41.377629 12190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:41.377830 12190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:41.377919 12190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:41.378063 12190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:41.378281 12190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:41.378309 12190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.378328 12190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:41.378355 12190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.383980 12190 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:44489
I20260926 13:04:41.384119 12317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:44489 every 8 connection(s)
I20260926 13:04:41.384505 12190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:04:41.387894 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.385567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:59059 (local address 127.2.86.62:45051)
0926 13:04:41.385632 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.385634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.385877 (+ 243us) server_negotiation.cc:407] Connection header received
0926 13:04:41.386407 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.386408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.386414 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.386472 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.387623 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:41.387624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:41.387625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:41.387628 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:41.387643 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:41.387648 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:41.387718 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:41.387822 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 13:04:41.387839 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:04:41.388116 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.385554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45051 (local address 127.2.86.1:59059)
0926 13:04:41.385743 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:41.385775 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:41.386300 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:41.386546 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:41.386555 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:41.386563 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:41.387440 (+ 877us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:41.387441 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:41.387483 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:41.387520 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:41.387746 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:41.387750 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:41.387754 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:41.387826 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 13:04:41.387891 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260926 13:04:41.388594 12318 heartbeater.cc:347] Connected to a master server at 127.2.86.62:45051
I20260926 13:04:41.388691 12318 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:41.388810 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12190
I20260926 13:04:41.388829 12318 heartbeater.cc:511] Master 127.2.86.62:45051 requested a full tablet report, sending...
I20260926 13:04:41.388931 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:41.389139 2392 external_mini_cluster.cc:1428] Setting key 0f2b53d3bb8113d8f3efd00ad736a81c
I20260926 13:04:41.389392 12130 ts_manager.cc:194] Registered new tserver with Master: 7b1dea6ebd7244a1b44229c33a6b7f81 (127.2.86.1:44489)
I20260926 13:04:41.390137 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--builtin_ntp_servers=127.2.86.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:04:41.469550 12321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:41.469753 12321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:41.469774 12321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:41.470870 12321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:41.470902 12321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:41.470927 12321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:41.472182 12321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:41.472358 12321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:41.472599 12321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:41.477411 12326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:41.477415 12329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.477532 12321 server_base.cc:1034] running on GCE node
W20260926 13:04:41.477706 12327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.478008 12321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:41.478195 12321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:41.479329 12321 hybrid_clock.cc:648] HybridClock initialized: now 1790427881479317 us; error 30 us; skew 500 ppm
I20260926 13:04:41.480595 12321 webserver.cc:460] Webserver started at http://127.2.86.2:43641/ using document root <none> and password file <none>
I20260926 13:04:41.480840 12321 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:41.480952 12321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:41.481098 12321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:41.482095 12321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "62541eaf29e342dcb6249d6f1059cfa2"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "066510f581018845027fb95b331d0acc"
server_key_iv: "465c0c3a986d03c2e93df018ab0a835f"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.482439 12321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "62541eaf29e342dcb6249d6f1059cfa2"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "066510f581018845027fb95b331d0acc"
server_key_iv: "465c0c3a986d03c2e93df018ab0a835f"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.483716 12321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:41.484411 12335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.484571 12321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:41.484663 12321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "62541eaf29e342dcb6249d6f1059cfa2"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "066510f581018845027fb95b331d0acc"
server_key_iv: "465c0c3a986d03c2e93df018ab0a835f"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.484743 12321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:41.487931 12321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:41.488173 12321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:41.488303 12321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:41.488494 12321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:41.488772 12321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:41.488837 12321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.488886 12321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:41.488916 12321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.494465 12321 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:34335
I20260926 13:04:41.494621 12448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:34335 every 8 connection(s)
I20260926 13:04:41.494964 12321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:41.498263 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.495678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35029 (local address 127.2.86.62:45051)
0926 13:04:41.495740 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.495742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.496031 (+ 289us) server_negotiation.cc:407] Connection header received
0926 13:04:41.496571 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.496573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.496580 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.496661 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.498072 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:41.498073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:41.498073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:41.498078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:41.498093 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:41.498098 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:41.498153 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:41.498199 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 13:04:41.498211 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:41.498471 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.495664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45051 (local address 127.2.86.2:35029)
0926 13:04:41.495866 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:41.495907 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:41.496481 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:41.496747 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:41.496751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:41.496760 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:41.497933 (+ 1173us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:41.497934 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:41.497960 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:41.497987 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:41.498154 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:41.498156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:41.498158 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:41.498207 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:04:41.498250 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260926 13:04:41.498936 12449 heartbeater.cc:347] Connected to a master server at 127.2.86.62:45051
I20260926 13:04:41.499027 12449 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:41.499130 12449 heartbeater.cc:511] Master 127.2.86.62:45051 requested a full tablet report, sending...
I20260926 13:04:41.499476 12129 ts_manager.cc:194] Registered new tserver with Master: 62541eaf29e342dcb6249d6f1059cfa2 (127.2.86.2:34335)
I20260926 13:04:41.504469 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12321
I20260926 13:04:41.504554 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:41.504757 2392 external_mini_cluster.cc:1428] Setting key 2c4f3adfab2ba26f28559371193720e6
I20260926 13:04:41.505611 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--builtin_ntp_servers=127.2.86.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 13:04:41.586248 12452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:41.586450 12452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:41.586473 12452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:41.587706 12452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:41.587761 12452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:41.587812 12452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:41.589294 12452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:41.589471 12452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:41.589697 12452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:41.594411 12460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:41.594506 12458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.594615 12452 server_base.cc:1034] running on GCE node
W20260926 13:04:41.594506 12457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:41.594872 12452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:41.595109 12452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:41.596273 12452 hybrid_clock.cc:648] HybridClock initialized: now 1790427881596263 us; error 42 us; skew 500 ppm
I20260926 13:04:41.597472 12452 webserver.cc:460] Webserver started at http://127.2.86.3:34787/ using document root <none> and password file <none>
I20260926 13:04:41.597702 12452 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:41.597769 12452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:41.597918 12452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:41.599018 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "2d2c49dfd66d42de8b8b5167b37c39d6"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "84a11cf6fbdea54087c4f89200016c0d"
server_key_iv: "c9af73d38cfe8169c3211655b44e07b0"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.599370 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "2d2c49dfd66d42de8b8b5167b37c39d6"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "84a11cf6fbdea54087c4f89200016c0d"
server_key_iv: "c9af73d38cfe8169c3211655b44e07b0"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.600703 12452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:04:41.601488 12466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.601662 12452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:04:41.601758 12452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "2d2c49dfd66d42de8b8b5167b37c39d6"
format_stamp: "Formatted at 2026-09-26 13:04:41 on dist-test-slave-j4lq"
server_key: "84a11cf6fbdea54087c4f89200016c0d"
server_key_iv: "c9af73d38cfe8169c3211655b44e07b0"
server_key_version: "encryptionkey@0"
I20260926 13:04:41.601842 12452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:41.605053 12452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:41.605314 12452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:41.605459 12452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:41.605665 12452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:41.605970 12452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:41.606025 12452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.606065 12452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:41.606091 12452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:41.611565 12452 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38565
I20260926 13:04:41.611630 12579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38565 every 8 connection(s)
I20260926 13:04:41.611915 12452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:04:41.614678 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.612606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:45877 (local address 127.2.86.62:45051)
0926 13:04:41.612672 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.612674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.612898 (+ 224us) server_negotiation.cc:407] Connection header received
0926 13:04:41.613384 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.613385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.613391 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.613444 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.614472 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:41.614473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:41.614474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:04:41.614478 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:41.614491 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:04:41.614495 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:41.614535 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:41.614614 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 13:04:41.614627 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:04:41.614841 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:41.612605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45051 (local address 127.2.86.3:45877)
0926 13:04:41.612769 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:41.612799 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:41.613295 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:41.613495 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:41.613498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:41.613506 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:41.614329 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:04:41.614330 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:41.614354 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:04:41.614379 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:41.614547 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:41.614551 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:41.614553 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:41.614611 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 13:04:41.614657 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260926 13:04:41.615334 12580 heartbeater.cc:347] Connected to a master server at 127.2.86.62:45051
I20260926 13:04:41.615401 12580 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:41.615499 12580 heartbeater.cc:511] Master 127.2.86.62:45051 requested a full tablet report, sending...
I20260926 13:04:41.615859 12129 ts_manager.cc:194] Registered new tserver with Master: 2d2c49dfd66d42de8b8b5167b37c39d6 (127.2.86.3:38565)
I20260926 13:04:41.620071 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12452
I20260926 13:04:41.620146 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:41.620332 2392 external_mini_cluster.cc:1428] Setting key ae8b36dcd1f48f6aadeed2b82a2b4627
I20260926 13:04:41.621268 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 13:04:41.623091 12169 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:04:41.622553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.2.86.62:45051)
0926 13:04:41.622613 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:41.622614 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:41.622739 (+ 125us) server_negotiation.cc:407] Connection header received
0926 13:04:41.622766 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:41.622767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:41.622772 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:41.622828 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:41.623012 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50550: BlockingRecv error: recv got EOF from 127.0.0.1:50550 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:04:41.623103 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12190
I20260926 13:04:41.627517 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12321
I20260926 13:04:41.631970 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12452
I20260926 13:04:41.636432 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12099
2026-09-26T13:04:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (528 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:41 dist-test-slave-j4lq krb5kdc[12590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:41 dist-test-slave-j4lq krb5kdc[12590](info): set up 2 sockets
Sep 26 13:04:41 dist-test-slave-j4lq krb5kdc[12590](info): commencing operation
krb5kdc: starting...
W20260926 13:04:42.661789 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:42 dist-test-slave-j4lq krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:04:42.776641 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46707
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:46707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:42.859287 12603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:42.859495 12603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:42.859519 12603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:42.860595 12603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:04:42.860631 12603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:42.860646 12603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:42.860658 12603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:04:42.860669 12603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:04:42.862142 12603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:46707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:42.862335 12603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:42.862507 12603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:42.867112 12611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:42.867107 12609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:42.867111 12608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:42.867205 12603 server_base.cc:1034] running on GCE node
I20260926 13:04:42.867519 12603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:42.867740 12603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:42.868930 12603 hybrid_clock.cc:648] HybridClock initialized: now 1790427882868914 us; error 29 us; skew 500 ppm
Sep 26 13:04:42 dist-test-slave-j4lq krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427882, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:42.871274 12603 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:04:42.871744 12603 webserver.cc:460] Webserver started at http://127.2.86.62:46139/ using document root <none> and password file <none>
I20260926 13:04:42.871971 12603 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:42.872037 12603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:42.872177 12603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:42.873303 12603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "53f383d77c384c51893451066ae3286a"
format_stamp: "Formatted at 2026-09-26 13:04:42 on dist-test-slave-j4lq"
server_key: "139611fa2b42f0c069f8fbbb8909bd90"
server_key_iv: "9479cdb39d471735b1d86a505bd46608"
server_key_version: "encryptionkey@0"
I20260926 13:04:42.873704 12603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "53f383d77c384c51893451066ae3286a"
format_stamp: "Formatted at 2026-09-26 13:04:42 on dist-test-slave-j4lq"
server_key: "139611fa2b42f0c069f8fbbb8909bd90"
server_key_iv: "9479cdb39d471735b1d86a505bd46608"
server_key_version: "encryptionkey@0"
I20260926 13:04:42.874979 12603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:42.875648 12618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:42.875813 12603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:42.875905 12603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "53f383d77c384c51893451066ae3286a"
format_stamp: "Formatted at 2026-09-26 13:04:42 on dist-test-slave-j4lq"
server_key: "139611fa2b42f0c069f8fbbb8909bd90"
server_key_iv: "9479cdb39d471735b1d86a505bd46608"
server_key_version: "encryptionkey@0"
I20260926 13:04:42.875983 12603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:42.888550 12603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:42.890466 12603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:42.890625 12603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:42.894521 12603 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:46707
I20260926 13:04:42.894554 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:46707 every 8 connection(s)
I20260926 13:04:42.894879 12603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:04:42.895815 12671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:42.898124 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: Bootstrap starting.
I20260926 13:04:42.898679 12671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:42.898967 12671 log.cc:824] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:42.899659 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: No bootstrap required, opened a new log
I20260926 13:04:42.900763 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12603
I20260926 13:04:42.900859 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:04:42.901065 2392 external_mini_cluster.cc:1428] Setting key 39bc3bd00168daea43d2d191a32397ba
I20260926 13:04:42.901818 12671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } }
I20260926 13:04:42.901973 12671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:42.901995 12671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f383d77c384c51893451066ae3286a, State: Initialized, Role: FOLLOWER
I20260926 13:04:42.902125 12671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } }
I20260926 13:04:42.902223 12671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:04:42.902268 12671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:04:42.902315 12671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:42.903221 12671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } }
I20260926 13:04:42.903311 12671 leader_election.cc:302] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f383d77c384c51893451066ae3286a; no voters:
I20260926 13:04:42.903487 12671 leader_election.cc:288] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:04:42.903529 12676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:42.903676 12676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [term 1 LEADER]: Becoming Leader. State: Replica: 53f383d77c384c51893451066ae3286a, State: Running, Role: LEADER
I20260926 13:04:42.903756 12671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:04:42.903811 12676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } }
I20260926 13:04:42.904942 12677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f383d77c384c51893451066ae3286a. Latest consensus state: current_term: 1 leader_uuid: "53f383d77c384c51893451066ae3286a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } } }
I20260926 13:04:42.905027 12677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [sys.catalog]: This master's current role is: LEADER
I20260926 13:04:42.905499 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f383d77c384c51893451066ae3286a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f383d77c384c51893451066ae3286a" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46707 } } }
W20260926 13:04:42.905581 12691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 13:04:42.905609 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a [sys.catalog]: This master's current role is: LEADER
W20260926 13:04:42.905714 12691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:04:42.905843 12692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:04:42.906010 12692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:04:42.907318 12692 catalog_manager.cc:1347] Generated new cluster ID: e57e26d803924f2d8639960198797e6a
I20260926 13:04:42.907368 12692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:04:42 dist-test-slave-j4lq krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:42.911482 12692 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:04:42.911931 12692 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:04:42.912994 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:42.901608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.2.86.62:46707)
0926 13:04:42.901780 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:42.901788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:42.901817 (+ 29us) server_negotiation.cc:407] Connection header received
0926 13:04:42.901915 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:42.901921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:42.901964 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:42.902144 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:42.902800 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:42.903959 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:42.905209 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:42.905816 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:42.905927 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:42.908754 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:42.908758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:42.908758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:42.908770 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:42.910801 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:42.911191 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:42.911195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:42.911195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:42.911314 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:42.911536 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:42.911537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:42.911537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:42.911800 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:42.911943 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:42.912431 (+ 488us) server_negotiation.cc:299] Negotiation successful
0926 13:04:42.912583 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260926 13:04:42.919934 12692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: Generated new TSK 0
I20260926 13:04:42.920091 12692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:04:42.947829 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:43.026903 12697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:43.027133 12697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:43.027174 12697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:43.028352 12697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:43.028412 12697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:43.028462 12697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:04:43.029922 12697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:43.030169 12697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:43.030388 12697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:43.035308 12703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:43.035429 12697 server_base.cc:1034] running on GCE node
W20260926 13:04:43.035476 12702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:43.035344 12705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:43.035723 12697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:43.035970 12697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:43.037125 12697 hybrid_clock.cc:648] HybridClock initialized: now 1790427883037118 us; error 37 us; skew 500 ppm
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:43.039463 12697 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:04:43.039856 12697 webserver.cc:460] Webserver started at http://127.2.86.1:36963/ using document root <none> and password file <none>
I20260926 13:04:43.040077 12697 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:43.040143 12697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:43.040287 12697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:43.041338 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "6586b49a599341cc9f784ffde1e17f7b"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "d97c9293d54209b74a9e4ac42de04aa2"
server_key_iv: "df40acf5c722ee173f36f1cff58c7b51"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.041695 12697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "6586b49a599341cc9f784ffde1e17f7b"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "d97c9293d54209b74a9e4ac42de04aa2"
server_key_iv: "df40acf5c722ee173f36f1cff58c7b51"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.043053 12697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:43.043702 12712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.043856 12697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:43.043958 12697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "6586b49a599341cc9f784ffde1e17f7b"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "d97c9293d54209b74a9e4ac42de04aa2"
server_key_iv: "df40acf5c722ee173f36f1cff58c7b51"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.044025 12697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:43.054152 12697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:43.056315 12697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:43.056464 12697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:43.056689 12697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:43.057046 12697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:43.057097 12697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.057137 12697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:43.057164 12697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.062824 12697 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:46321
I20260926 13:04:43.062928 12825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:46321 every 8 connection(s)
I20260926 13:04:43.063244 12697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:43.072098 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.064031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:53117 (local address 127.2.86.62:46707)
0926 13:04:43.064101 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.064103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.064317 (+ 214us) server_negotiation.cc:407] Connection header received
0926 13:04:43.064800 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.064801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.064810 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.064909 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:43.065826 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.066317 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.067292 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.067482 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.067532 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.069597 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:43.069599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:43.069599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:43.069605 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:43.070867 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.071245 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.071245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.071246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.071291 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.071507 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.071508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.071508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.071621 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:43.071669 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.071901 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.071979 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:04:43.072297 12828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.064022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:46707 (local address 127.2.86.1:53117)
0926 13:04:43.064196 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.064239 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.064718 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.064966 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.064970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.065549 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:43.065748 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.065753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.066396 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.066396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.067216 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.067218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.067549 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.067550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.067655 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.068131 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:43.068139 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:43.069519 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:43.070952 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.070958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.070959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.071183 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.071354 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.071354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.071355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.071452 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.071676 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:43.071681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:43.071789 (+ 108us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.071875 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.072004 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260926 13:04:43.072702 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12697
I20260926 13:04:43.072844 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:04:43.072947 12826 heartbeater.cc:347] Connected to a master server at 127.2.86.62:46707
I20260926 13:04:43.073060 12826 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:43.073082 2392 external_mini_cluster.cc:1428] Setting key f356b8b9ff68239d60b460ee07ca6088
I20260926 13:04:43.073233 12826 heartbeater.cc:511] Master 127.2.86.62:46707 requested a full tablet report, sending...
I20260926 13:04:43.073889 12634 ts_manager.cc:194] Registered new tserver with Master: 6586b49a599341cc9f784ffde1e17f7b (127.2.86.1:46321)
I20260926 13:04:43.074512 12634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:53117
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:04:43.095880 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:43.175028 12831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:43.175215 12831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:43.175236 12831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:43.176381 12831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:43.176421 12831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:43.176447 12831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:04:43.177894 12831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:43.178153 12831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:43.178380 12831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:43.183473 12836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:43.183501 12837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:43.183511 12839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:43.183725 12831 server_base.cc:1034] running on GCE node
I20260926 13:04:43.183964 12831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:43.184226 12831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:43.185412 12831 hybrid_clock.cc:648] HybridClock initialized: now 1790427883185407 us; error 36 us; skew 500 ppm
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:43.187944 12831 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:04:43.188405 12831 webserver.cc:460] Webserver started at http://127.2.86.2:43687/ using document root <none> and password file <none>
I20260926 13:04:43.188606 12831 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:43.188645 12831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:43.188731 12831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:43.189807 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "26c8f77969844320b72bd5bad4c62c6c"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "459e884504b40935cf110c15e85fc562"
server_key_iv: "c610b8aaa338dfc00efef91cf812a05f"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.190097 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "26c8f77969844320b72bd5bad4c62c6c"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "459e884504b40935cf110c15e85fc562"
server_key_iv: "c610b8aaa338dfc00efef91cf812a05f"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.191309 12831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:04:43.192019 12846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.192170 12831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:04:43.192253 12831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "26c8f77969844320b72bd5bad4c62c6c"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "459e884504b40935cf110c15e85fc562"
server_key_iv: "c610b8aaa338dfc00efef91cf812a05f"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.192306 12831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:43.205895 12831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:43.207757 12831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:43.207861 12831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:43.208037 12831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:43.208338 12831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:43.208372 12831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.208393 12831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:43.208447 12831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.214550 12831 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:43851
I20260926 13:04:43.214605 12959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:43851 every 8 connection(s)
I20260926 13:04:43.214910 12831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:04:43.220141 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12831
I20260926 13:04:43.220252 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:04:43.220460 2392 external_mini_cluster.cc:1428] Setting key 6fb4a26f2e9e231fe53b263fc275ef48
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:43.224607 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.215652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:46707 (local address 127.2.86.2:44263)
0926 13:04:43.215818 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.215855 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.216315 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.216526 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.216529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.217090 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:43.217274 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.217278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.217922 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.217923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.218711 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.218712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.219034 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.219035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.219136 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.219671 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:43.219680 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:43.221322 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:43.223147 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.223153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.223156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.223470 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.223665 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.223666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.223666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.223815 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.224123 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:43.224128 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:43.224245 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.224329 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.224452 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260926 13:04:43.224603 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.215671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:44263 (local address 127.2.86.62:46707)
0926 13:04:43.215740 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.215742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.215953 (+ 211us) server_negotiation.cc:407] Connection header received
0926 13:04:43.216410 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.216411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.216421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.216473 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:43.217349 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.217834 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.218784 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.218966 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.219016 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.221424 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:43.221425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:43.221425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:43.221434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:43.223067 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.223548 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.223548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.223549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.223601 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.223892 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.223912 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.223913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.224055 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:43.224129 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.224376 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.224464 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:04:43.225265 12960 heartbeater.cc:347] Connected to a master server at 127.2.86.62:46707
I20260926 13:04:43.225371 12960 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:43.225561 12960 heartbeater.cc:511] Master 127.2.86.62:46707 requested a full tablet report, sending...
I20260926 13:04:43.225953 12634 ts_manager.cc:194] Registered new tserver with Master: 26c8f77969844320b72bd5bad4c62c6c (127.2.86.2:43851)
I20260926 13:04:43.226505 12634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:44263
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:04:43.247789 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:04:43.328101 12965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:04:43.328292 12965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:04:43.328320 12965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:04:43.329506 12965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:04:43.329545 12965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:04:43.329571 12965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:04:43.330893 12965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:46707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:04:43.331060 12965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:04:43.331218 12965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:04:43.336143 12970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:43.336228 12965 server_base.cc:1034] running on GCE node
W20260926 13:04:43.336315 12971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:04:43.336169 12973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:04:43.336540 12965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:04:43.336844 12965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:04:43.338029 12965 hybrid_clock.cc:648] HybridClock initialized: now 1790427883338008 us; error 38 us; skew 500 ppm
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:04:43.340430 12965 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:04:43.341035 12965 webserver.cc:460] Webserver started at http://127.2.86.3:39085/ using document root <none> and password file <none>
I20260926 13:04:43.341300 12965 fs_manager.cc:359] Metadata directory not provided
I20260926 13:04:43.341370 12965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:04:43.341526 12965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:04:43.342626 12965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "39b399805c1f4790a3db5dc9d5e4952f"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "5573f248e587cbafddd1dd39bdc840b9"
server_key_iv: "f9abec165b5b6c9dd85864550e8d86f7"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.342962 12965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "39b399805c1f4790a3db5dc9d5e4952f"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "5573f248e587cbafddd1dd39bdc840b9"
server_key_iv: "f9abec165b5b6c9dd85864550e8d86f7"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.344169 12965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:04:43.344841 12980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.345062 12965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:04:43.345151 12965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "39b399805c1f4790a3db5dc9d5e4952f"
format_stamp: "Formatted at 2026-09-26 13:04:43 on dist-test-slave-j4lq"
server_key: "5573f248e587cbafddd1dd39bdc840b9"
server_key_iv: "f9abec165b5b6c9dd85864550e8d86f7"
server_key_version: "encryptionkey@0"
I20260926 13:04:43.345237 12965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:04:43.356278 12965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:04:43.358103 12965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:04:43.358254 12965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:04:43.358460 12965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:04:43.358757 12965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:04:43.358812 12965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.358853 12965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:04:43.358880 12965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:04:43.364567 12965 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:39921
I20260926 13:04:43.364617 13093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:39921 every 8 connection(s)
I20260926 13:04:43.365015 12965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427883, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:04:43.372081 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 12965
I20260926 13:04:43.372197 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:04:43.372400 2392 external_mini_cluster.cc:1428] Setting key 7f59d862cfade185f7fbf71397e26a93
I20260926 13:04:43.374408 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.365770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:48493 (local address 127.2.86.62:46707)
0926 13:04:43.365838 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.365840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.366055 (+ 215us) server_negotiation.cc:407] Connection header received
0926 13:04:43.366574 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.366575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.366587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.366640 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:43.367555 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.368072 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.369081 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.369296 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.369384 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.371496 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:43.371497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:43.371497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:43.371506 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:43.373319 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.373676 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.373677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.373677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.373728 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.373925 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.373926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.373926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.374024 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:43.374073 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.374286 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.374345 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:04:43.374572 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.365739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:46707 (local address 127.2.86.3:48493)
0926 13:04:43.365928 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.365971 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.366469 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.366709 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.366713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.367266 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:04:43.367453 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.367458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.368164 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.368165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.368991 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.368993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.369391 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.369392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.369519 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.370042 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:04:43.370050 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:04:43.371394 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:04:43.373394 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.373406 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.373407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.373615 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.373786 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:04:43.373786 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:04:43.373787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:04:43.373875 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:04:43.374076 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:04:43.374080 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:04:43.374184 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.374262 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.374372 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260926 13:04:43.374972 13094 heartbeater.cc:347] Connected to a master server at 127.2.86.62:46707
I20260926 13:04:43.375069 13094 heartbeater.cc:464] Registering TS with master...
I20260926 13:04:43.375231 13094 heartbeater.cc:511] Master 127.2.86.62:46707 requested a full tablet report, sending...
I20260926 13:04:43.375599 12634 ts_manager.cc:194] Registered new tserver with Master: 39b399805c1f4790a3db5dc9d5e4952f (127.2.86.3:39921)
I20260926 13:04:43.375967 12634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:48493
I20260926 13:04:43.384167 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:04:43.391734 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.385490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54930 (local address 127.2.86.62:46707)
0926 13:04:43.385569 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.385571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.385644 (+ 73us) server_negotiation.cc:407] Connection header received
0926 13:04:43.385721 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.385722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.385732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.385799 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:04:43.386386 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.386879 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.387623 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.387820 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.387904 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.389127 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:04:43.389128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:04:43.389128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:04:43.389135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:04:43.390411 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.390879 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.390880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.390880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.390954 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:04:43.391174 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:04:43.391175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:04:43.391175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:04:43.391392 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:04:43.391468 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.391618 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.391675 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:04:43.393208 12634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:04:43.394735 12634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:04:43.401012 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.396846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:43851 (local address 127.0.0.1:42738)
0926 13:04:43.397014 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.397053 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.397114 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.397442 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.397444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.397446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:43.397530 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.397532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.398624 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.398625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.399790 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.399792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.400692 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.400769 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.400783 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.400851 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.400939 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260926 13:04:43.401283 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.396951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.2.86.2:43851)
0926 13:04:43.397254 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.397259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.397269 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:43.397306 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.397309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.397330 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.397423 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:43.397826 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.398518 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.399887 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.400603 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400681 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.400738 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.401161 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.401207 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":98,"threads_started":1}
I20260926 13:04:43.401298 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.397044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.2.86.3:39921)
0926 13:04:43.397559 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.397563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.397571 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:04:43.397610 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.397613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.397627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.397728 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:43.397807 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.398877 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400089 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.400789 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400858 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.400959 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.401045 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.401229 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":90,"threads_started":1}
I20260926 13:04:43.401149 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.396898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:39921 (local address 127.0.0.1:54828)
0926 13:04:43.397133 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.397145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.397199 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.397725 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.397726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.397727 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:43.397761 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.397763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.398974 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.398975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.400018 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.400019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.400898 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.400966 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.400971 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.401030 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.401072 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":68,"threads_started":1}
I20260926 13:04:43.402061 13028 tablet_service.cc:1467] Processing CreateTablet for tablet f2aafa44ca404ee4862127db99de27c1 (DEFAULT_TABLE table=test-table [id=302b736632b144fe918819aa651d003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:43.402081 12894 tablet_service.cc:1467] Processing CreateTablet for tablet f2aafa44ca404ee4862127db99de27c1 (DEFAULT_TABLE table=test-table [id=302b736632b144fe918819aa651d003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:43.402175 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.397057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:46321 (local address 127.0.0.1:33850)
0926 13:04:43.397273 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.397283 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.397329 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.397598 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.397599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.397600 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:43.397641 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.397643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.400133 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.400880 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.400901 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.401914 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.401915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.401983 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.401987 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.402069 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.402110 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260926 13:04:43.402195 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.397269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.2.86.1:46321)
0926 13:04:43.397450 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.397454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.397464 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:04:43.397500 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.397502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.397517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.397599 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:43.397748 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.398867 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.400981 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.401830 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.401915 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.401968 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.402085 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.402121 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260926 13:04:43.402410 12894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2aafa44ca404ee4862127db99de27c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:43.402410 13028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2aafa44ca404ee4862127db99de27c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:43.402758 12760 tablet_service.cc:1467] Processing CreateTablet for tablet f2aafa44ca404ee4862127db99de27c1 (DEFAULT_TABLE table=test-table [id=302b736632b144fe918819aa651d003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:04:43.402989 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2aafa44ca404ee4862127db99de27c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:04:43.405709 13108 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Bootstrap starting.
I20260926 13:04:43.405794 13110 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Bootstrap starting.
I20260926 13:04:43.406308 13109 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Bootstrap starting.
I20260926 13:04:43.406374 13110 tablet_bootstrap.cc:654] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:43.406373 13108 tablet_bootstrap.cc:654] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:43.406908 13109 tablet_bootstrap.cc:654] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Neither blocks nor log segments found. Creating new log.
I20260926 13:04:43.406962 13110 log.cc:824] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:43.406957 13108 log.cc:824] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:43.407490 13109 log.cc:824] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:04:43.407742 13108 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: No bootstrap required, opened a new log
I20260926 13:04:43.407819 13108 ts_tablet_manager.cc:1397] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:43.408144 13109 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: No bootstrap required, opened a new log
I20260926 13:04:43.408195 13109 ts_tablet_manager.cc:1397] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:04:43.409446 13108 raft_consensus.cc:348] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.409590 13108 raft_consensus.cc:374] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:43.409632 13108 raft_consensus.cc:729] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6586b49a599341cc9f784ffde1e17f7b, State: Initialized, Role: FOLLOWER
I20260926 13:04:43.409735 13108 consensus_queue.cc:260] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.409649 13109 raft_consensus.cc:348] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.409813 13109 raft_consensus.cc:374] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:43.409873 13109 raft_consensus.cc:729] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c8f77969844320b72bd5bad4c62c6c, State: Initialized, Role: FOLLOWER
I20260926 13:04:43.409994 13108 ts_tablet_manager.cc:1428] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:43.410033 12826 heartbeater.cc:503] Master 127.2.86.62:46707 was elected leader, sending a full tablet report...
I20260926 13:04:43.410008 13109 consensus_queue.cc:260] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.410276 13109 ts_tablet_manager.cc:1428] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:43.410316 12960 heartbeater.cc:503] Master 127.2.86.62:46707 was elected leader, sending a full tablet report...
I20260926 13:04:43.410743 13110 tablet_bootstrap.cc:492] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: No bootstrap required, opened a new log
I20260926 13:04:43.410816 13110 ts_tablet_manager.cc:1397] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 13:04:43.412200 13110 raft_consensus.cc:348] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.412336 13110 raft_consensus.cc:374] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:04:43.412381 13110 raft_consensus.cc:729] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b399805c1f4790a3db5dc9d5e4952f, State: Initialized, Role: FOLLOWER
I20260926 13:04:43.412493 13110 consensus_queue.cc:260] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.412761 13110 ts_tablet_manager.cc:1428] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:04:43.412812 13094 heartbeater.cc:503] Master 127.2.86.62:46707 was elected leader, sending a full tablet report...
W20260926 13:04:43.465480 12961 tablet.cc:2369] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:04:43.564028 12827 tablet.cc:2369] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:04:43.605161 13115 raft_consensus.cc:482] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:04:43.605288 13115 raft_consensus.cc:504] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.605945 13115 leader_election.cc:288] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39b399805c1f4790a3db5dc9d5e4952f (127.2.86.3:39921), 6586b49a599341cc9f784ffde1e17f7b (127.2.86.1:46321)
I20260926 13:04:43.608946 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.606261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:55901 (local address 127.2.86.3:39921)
0926 13:04:43.606301 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.606302 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.606333 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:04:43.606365 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.606366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.606378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.606479 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:43.606661 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.607160 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608189 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.608574 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608626 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.608658 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.608817 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.608855 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:04:43.608946 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.606214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:46753 (local address 127.2.86.1:46321)
0926 13:04:43.606259 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.606261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.606479 (+ 218us) server_negotiation.cc:407] Connection header received
0926 13:04:43.606524 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.606525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.606537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.606592 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:04:43.606770 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.607255 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608235 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.608579 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608665 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.608675 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.608820 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.608855 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:04:43.608956 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.606136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:39921 (local address 127.2.86.2:55901)
0926 13:04:43.606197 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.606211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.606257 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.606500 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.606501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.606502 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:43.606583 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.606586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.607243 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.607244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.608103 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.608105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.608660 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.608741 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.608743 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.608811 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.608856 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":51}
I20260926 13:04:43.609050 13117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.606152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:46321 (local address 127.2.86.2:46753)
0926 13:04:43.606384 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.606393 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:04:43.606467 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:04:43.606600 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:04:43.606600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:04:43.606602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:04:43.606687 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.606689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.607328 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.607329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.608120 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:04:43.608122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.608680 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.608681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:04:43.608742 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.608745 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:04:43.608812 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:04:43.608855 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":87,"thread_start_us":100,"threads_started":1}
I20260926 13:04:43.609341 13048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2aafa44ca404ee4862127db99de27c1" candidate_uuid: "26c8f77969844320b72bd5bad4c62c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b399805c1f4790a3db5dc9d5e4952f" is_pre_election: true
I20260926 13:04:43.609422 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2aafa44ca404ee4862127db99de27c1" candidate_uuid: "26c8f77969844320b72bd5bad4c62c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6586b49a599341cc9f784ffde1e17f7b" is_pre_election: true
I20260926 13:04:43.609504 13048 raft_consensus.cc:2454] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8f77969844320b72bd5bad4c62c6c in term 0.
I20260926 13:04:43.609586 12780 raft_consensus.cc:2454] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8f77969844320b72bd5bad4c62c6c in term 0.
I20260926 13:04:43.609704 12850 leader_election.cc:302] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c8f77969844320b72bd5bad4c62c6c, 39b399805c1f4790a3db5dc9d5e4952f; no voters:
I20260926 13:04:43.609820 13115 raft_consensus.cc:2781] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:04:43.609886 13115 raft_consensus.cc:482] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:04:43.609905 13115 raft_consensus.cc:3037] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:43.610540 13115 raft_consensus.cc:504] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.610684 13115 leader_election.cc:288] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [CANDIDATE]: Term 1 election: Requested vote from peers 39b399805c1f4790a3db5dc9d5e4952f (127.2.86.3:39921), 6586b49a599341cc9f784ffde1e17f7b (127.2.86.1:46321)
I20260926 13:04:43.610838 13048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2aafa44ca404ee4862127db99de27c1" candidate_uuid: "26c8f77969844320b72bd5bad4c62c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b399805c1f4790a3db5dc9d5e4952f"
I20260926 13:04:43.610924 13048 raft_consensus.cc:3037] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:43.610903 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2aafa44ca404ee4862127db99de27c1" candidate_uuid: "26c8f77969844320b72bd5bad4c62c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6586b49a599341cc9f784ffde1e17f7b"
I20260926 13:04:43.611011 12780 raft_consensus.cc:3037] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:04:43.611630 13048 raft_consensus.cc:2454] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8f77969844320b72bd5bad4c62c6c in term 1.
I20260926 13:04:43.611725 12780 raft_consensus.cc:2454] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8f77969844320b72bd5bad4c62c6c in term 1.
I20260926 13:04:43.611781 12850 leader_election.cc:302] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c8f77969844320b72bd5bad4c62c6c, 39b399805c1f4790a3db5dc9d5e4952f; no voters:
I20260926 13:04:43.611888 13115 raft_consensus.cc:2781] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:04:43.612004 13115 raft_consensus.cc:686] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 1 LEADER]: Becoming Leader. State: Replica: 26c8f77969844320b72bd5bad4c62c6c, State: Running, Role: LEADER
I20260926 13:04:43.612097 13115 consensus_queue.cc:237] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } }
I20260926 13:04:43.612929 12634 catalog_manager.cc:5495] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 26c8f77969844320b72bd5bad4c62c6c (127.2.86.2). New cstate: current_term: 1 leader_uuid: "26c8f77969844320b72bd5bad4c62c6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8f77969844320b72bd5bad4c62c6c" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 43851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 } health_report { overall_health: UNKNOWN } } }
W20260926 13:04:43.615515 13095 tablet.cc:2369] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:04:43.630832 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.628381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.2.86.2:43851)
0926 13:04:43.628478 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 13:04:43.628480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:04:43.628487 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:04:43.628520 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:04:43.628521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:04:43.628538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:04:43.628599 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:04:43.628729 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.629254 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.629962 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:04:43.630137 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:04:43.630185 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:04:43.630318 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:04:43.630412 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:04:43.630475 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:04:43.630690 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 13:04:43.630707 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 13:04:43.632673 13048 raft_consensus.cc:1261] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 1 FOLLOWER]: Refusing update from remote peer 26c8f77969844320b72bd5bad4c62c6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:43.632673 12780 raft_consensus.cc:1261] T f2aafa44ca404ee4862127db99de27c1 P 6586b49a599341cc9f784ffde1e17f7b [term 1 FOLLOWER]: Refusing update from remote peer 26c8f77969844320b72bd5bad4c62c6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:04:43.633085 13115 consensus_queue.cc:1035] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b399805c1f4790a3db5dc9d5e4952f" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:43.633234 13118 consensus_queue.cc:1035] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6586b49a599341cc9f784ffde1e17f7b" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 46321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:04:43.635531 13122 mvcc.cc:205] Tried to move back new op lower bound from 7333592611356332032 to 7333592611275595776. Current Snapshot: MvccSnapshot[applied={T|T < 7333592611356332032}]
I20260926 13:04:43.635677 13121 mvcc.cc:205] Tried to move back new op lower bound from 7333592611356332032 to 7333592611275595776. Current Snapshot: MvccSnapshot[applied={T|T < 7333592611356332032}]
I20260926 13:04:43.637284 13124 mvcc.cc:205] Tried to move back new op lower bound from 7333592611356332032 to 7333592611275595776. Current Snapshot: MvccSnapshot[applied={T|T < 7333592611356332032 or (T in {7333592611356332032})}]
I20260926 13:04:43.638521 12635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:43.638633 12635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:04:43.639369 12635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53f383d77c384c51893451066ae3286a: Sending DeleteTablet for 3 replicas of tablet f2aafa44ca404ee4862127db99de27c1
I20260926 13:04:43.639730 12894 tablet_service.cc:1514] Processing DeleteTablet for tablet f2aafa44ca404ee4862127db99de27c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42738
I20260926 13:04:43.639762 13028 tablet_service.cc:1514] Processing DeleteTablet for tablet f2aafa44ca404ee4862127db99de27c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:54828
I20260926 13:04:43.639757 12760 tablet_service.cc:1514] Processing DeleteTablet for tablet f2aafa44ca404ee4862127db99de27c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:04:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:33850
I20260926 13:04:43.640091 13132 tablet_replica.cc:330] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: stopping tablet replica
I20260926 13:04:43.640246 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12697
I20260926 13:04:43.642915 13132 raft_consensus.cc:2229] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:04:43.643059 13132 raft_consensus.cc:2258] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:43.643208 13131 tablet_replica.cc:330] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: stopping tablet replica
I20260926 13:04:43.643301 13131 raft_consensus.cc:2229] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:04:43.643402 13131 raft_consensus.cc:2258] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:04:43.643556 13132 ts_tablet_manager.cc:1905] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:43.643838 13131 ts_tablet_manager.cc:1905] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:04:43.645421 13132 ts_tablet_manager.cc:1918] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:43.645498 13132 log.cc:1196] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/f2aafa44ca404ee4862127db99de27c1
I20260926 13:04:43.645797 13132 ts_tablet_manager.cc:1939] T f2aafa44ca404ee4862127db99de27c1 P 39b399805c1f4790a3db5dc9d5e4952f: Deleting consensus metadata
I20260926 13:04:43.646116 13131 ts_tablet_manager.cc:1918] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:04:43.646235 13131 log.cc:1196] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/f2aafa44ca404ee4862127db99de27c1
I20260926 13:04:43.646206 12622 catalog_manager.cc:4841] TS 39b399805c1f4790a3db5dc9d5e4952f (127.2.86.3:39921): tablet f2aafa44ca404ee4862127db99de27c1 (table test-table [id=302b736632b144fe918819aa651d003e]) successfully deleted
I20260926 13:04:43.646541 13131 ts_tablet_manager.cc:1939] T f2aafa44ca404ee4862127db99de27c1 P 26c8f77969844320b72bd5bad4c62c6c: Deleting consensus metadata
I20260926 13:04:43.646981 12621 catalog_manager.cc:4841] TS 26c8f77969844320b72bd5bad4c62c6c (127.2.86.2:43851): tablet f2aafa44ca404ee4862127db99de27c1 (table test-table [id=302b736632b144fe918819aa651d003e]) successfully deleted
W20260926 13:04:43.647650 12621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:46321 (error 108)
I20260926 13:04:43.648205 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:04:43.648073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:46321 (local address 127.0.0.1:33852)
0926 13:04:43.648133 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 13:04:43.648171 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:46321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 13:04:43.648283 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12831
W20260926 13:04:43.649422 12621 catalog_manager.cc:4584] TS 6586b49a599341cc9f784ffde1e17f7b (127.2.86.1:46321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2aafa44ca404ee4862127db99de27c1: Network error: Client connection negotiation failed: client connection to 127.2.86.1:46321: connect: Connection refused (error 111)
I20260926 13:04:43.653177 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12965
I20260926 13:04:43.657918 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 12603
2026-09-26T13:04:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2022 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[13136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[13136](info): set up 2 sockets
Sep 26 13:04:43 dist-test-slave-j4lq krb5kdc[13136](info): commencing operation
krb5kdc: starting...
W20260926 13:04:44.686058 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:04:44 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:04:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangeradmin_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangeradmin_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangerlookup_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangerlookup_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
I20260926 13:04:44.846930 2392 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNbmtbn/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/postgres -l logfile start
2026-09-26 13:04:47.216 UTC [13166] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 13:04:47.216 UTC [13166] LOG: listening on IPv4 address "127.2.86.20", port 43551
2026-09-26 13:04:47.220 UTC [13166] LOG: listening on Unix socket "/tmp/.s.PGSQL.43551"
2026-09-26 13:04:47.249 UTC [13169] LOG: database system was shut down at 2026-09-26 13:04:45 UTC
2026-09-26 13:04:47.253 UTC [13166] LOG: database system is ready to accept connections
I20260926 13:04:48.193722 2392 mini_postgres.cc:96] Postgres bound to 43551
2026-09-26 13:04:48.199 UTC [13177] FATAL: database "slave" does not exist
127.2.86.20:43551 - accepting connections
I20260926 13:04:48.200380 2392 mini_ranger.cc:161] Starting Ranger...
I20260926 13:04:48.214531 2392 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 13:04:48.666986 2392 mini_ranger.cc:87] Created ranger Postgres database
I20260926 13:04:48.667078 2392 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin
2026-09-26 13:04:48,944 [I] DB FLAVOR :POSTGRES
2026-09-26 13:04:48,944 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:04:48,944 [I] Checking connection..
2026-09-26 13:04:48,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 13:04:49,183 [I] Checking connection passed.
2026-09-26 13:04:49,183 [I] --------- Verifying version history table ---------
2026-09-26 13:04:49,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:04:49,408 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:04:49,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:04:49,640 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:04:49,640 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 13:04:49,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 13:04:49.894 UTC [13241] WARNING: there is no transaction in progress
2026-09-26 13:04:49,900 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 13:04:49,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:04:50,133 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 13:04:50,133 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:04:50,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 13:04:50,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 13:04:50,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j4lq.c.gcp-upstream.internal','N') ;"
2026-09-26 13:04:50,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:04:51,067 [I] Table x_portal_user does not exist in database ranger
2026-09-26 13:04:51,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:04:51,293 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 13:04:51,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:04:51,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:04:51,793 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 13:04:51,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 13:04:52.734 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.750 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.766 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.781 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.799 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.819 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.883 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.889 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.903 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.913 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.923 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.936 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.946 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.953 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.959 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:52.966 UTC [13349] WARNING: there is no transaction in progress
2026-09-26 13:04:53,580 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 13:04:53,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 13:04:53,801 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 13:04:53,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:04:54,056 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 13:04:54,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:04:54,294 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 13:04:54,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:04:54,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:04:54,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j4lq.c.gcp-upstream.internal';"
2026-09-26 13:04:55,008 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 13:04:55,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 13:04:55,228 [I] DB_PATCHES have already been applied
I20260926 13:04:55.234043 2392 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNbmtbn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 13:04:55.234102 2392 mini_ranger.cc:193] Using host: 127.2.86.20
I20260926 13:04:55.237001 2392 mini_ranger.cc:227] Ranger admin URL: http://127.2.86.20:47119
Sep 26, 2026 1:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 1:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 1:04:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 1:04:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 1:04:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 13:04:56.352022 2392 mini_ranger.cc:160] Time spent starting Ranger: real 8.152s user 0.000s sys 0.008s
Sep 26, 2026 1:05:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 1:05:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Sep 26, 2026 1:05:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 1:05:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 1:05:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 13:05:18.306087 2392 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangerkms_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/rangerkms_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.86.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.86.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 13:05:18 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427918, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 13:05:18.392472 2392 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 13:05:18.401062 2392 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 13:05:19.239615 2392 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 13:05:19.239681 2392 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms
I20260926 13:05:19.239694 2392 mini_ranger_kms.cc:227] Using postgres at 127.2.86.20:43551
2026-09-26 13:05:19,420 [I] DB FLAVOR :POSTGRES
2026-09-26 13:05:19,421 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:05:19,421 [I] Checking connection
2026-09-26 13:05:19,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 13:05:19,660 [I] connection success
2026-09-26 13:05:19,660 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 13:05:19,660 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 13:05:19,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 13:05:19,914 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 13:05:19,915 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:05:19,915 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 13:05:19,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:43551/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 13:05:20,166 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 13:05:20.656374 2392 mini_ranger_kms.cc:310] Created kms service
I20260926 13:05:20.829406 2392 mini_ranger_kms.cc:326] Created kudu user
I20260926 13:05:20.871357 2392 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 13:05:20 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427918, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:21.225903 2392 mini_ranger_kms.cc:384] Added ranger policy
I20260926 13:05:21.226173 2392 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 13:05:21.226199 2392 mini_ranger_kms.cc:240] Using host: 127.2.86.20
I20260926 13:05:21.229912 2392 mini_ranger_kms.cc:276] Ranger KMS PID: 13529
I20260926 13:05:21.230034 2392 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.86.20:60111
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 1:05:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 13:05:23.455193 2392 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.063s user 0.012s sys 0.151s
I20260926 13:05:23.455310 2392 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 13:05:23.455394 2392 mini_ranger_kms.cc:401] 127.2.86.20:60111/kms/v1/keys
Sep 26 13:05:23 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790427923, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 13:05:24 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427923, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
Sep 26 13:05:24 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427923, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
Sep 26 13:05:26 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:05:26.604511 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40045
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40045
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:60111/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:05:26.688789 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:05:26.689077 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:05:26.689100 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:05:26.690237 13580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:05:26.690279 13580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:05:26.690294 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:05:26.690305 13580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:05:26.690317 13580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:05:26.691839 13580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.86.20:60111/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40045
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:05:26.692081 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:05:26.692317 13580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:05:26.698269 13580 server_base.cc:1034] running on GCE node
W20260926 13:05:26.698346 13588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:26.698237 13586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:26.698222 13585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:05:26.698840 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:05:26.699163 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:05:26.700347 13580 hybrid_clock.cc:648] HybridClock initialized: now 1790427926700321 us; error 43 us; skew 500 ppm
Sep 26 13:05:26 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:05:26.706820 13580 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:05:26.707352 13580 webserver.cc:460] Webserver started at http://127.2.86.62:44541/ using document root <none> and password file <none>
I20260926 13:05:26.707590 13580 fs_manager.cc:359] Metadata directory not provided
I20260926 13:05:26.707652 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:05:26.707816 13580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:05:26 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427926, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:26.755146 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "8111885e336c489887eea0c9a2419aca"
format_stamp: "Formatted at 2026-09-26 13:05:26 on dist-test-slave-j4lq"
server_key: "6edcbb6daf686a0bd2bf447a7fdd5b04"
server_key_iv: "7488b614467b362b50ad9854f8453cd7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:26.755611 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "8111885e336c489887eea0c9a2419aca"
format_stamp: "Formatted at 2026-09-26 13:05:26 on dist-test-slave-j4lq"
server_key: "6edcbb6daf686a0bd2bf447a7fdd5b04"
server_key_iv: "7488b614467b362b50ad9854f8453cd7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:26.757565 13580 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:05:26.769596 13595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:26.769853 13580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:05:26.770030 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "8111885e336c489887eea0c9a2419aca"
format_stamp: "Formatted at 2026-09-26 13:05:26 on dist-test-slave-j4lq"
server_key: "6edcbb6daf686a0bd2bf447a7fdd5b04"
server_key_iv: "7488b614467b362b50ad9854f8453cd7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:26.770140 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:05:26.785573 13580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:05:26.788033 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:05:26.788213 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:05:26.793526 13580 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40045
I20260926 13:05:26.793550 13647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40045 every 8 connection(s)
I20260926 13:05:26.794015 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:05:26.794701 13648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:05:26.797183 13648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: Bootstrap starting.
I20260926 13:05:26.798208 13648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: Neither blocks nor log segments found. Creating new log.
I20260926 13:05:26.798484 13648 log.cc:824] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: Log is configured to *not* fsync() on all Append() calls
I20260926 13:05:26.799185 13648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: No bootstrap required, opened a new log
I20260926 13:05:26.800505 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 13580
I20260926 13:05:26.800601 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:05:26.800784 2392 external_mini_cluster.cc:1428] Setting key 44f6914785424021f8956e5055f7712e
I20260926 13:05:26.801602 13648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } }
I20260926 13:05:26.801769 13648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:05:26.801831 13648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8111885e336c489887eea0c9a2419aca, State: Initialized, Role: FOLLOWER
I20260926 13:05:26.802011 13648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } }
I20260926 13:05:26.802122 13648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:05:26.802178 13648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:05:26.802254 13648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:05:26.802877 13648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } }
I20260926 13:05:26.802999 13648 leader_election.cc:302] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8111885e336c489887eea0c9a2419aca; no voters:
I20260926 13:05:26.803190 13648 leader_election.cc:288] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:05:26.803246 13653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:05:26.803455 13653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [term 1 LEADER]: Becoming Leader. State: Replica: 8111885e336c489887eea0c9a2419aca, State: Running, Role: LEADER
I20260926 13:05:26.803591 13648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:05:26.803604 13653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } }
I20260926 13:05:26.804065 13653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8111885e336c489887eea0c9a2419aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } } }
I20260926 13:05:26.804188 13653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [sys.catalog]: This master's current role is: LEADER
I20260926 13:05:26.804339 13653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8111885e336c489887eea0c9a2419aca. Latest consensus state: current_term: 1 leader_uuid: "8111885e336c489887eea0c9a2419aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8111885e336c489887eea0c9a2419aca" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40045 } } }
I20260926 13:05:26.804426 13653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca [sys.catalog]: This master's current role is: LEADER
I20260926 13:05:26.805073 13648 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:05:26.809152 13662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:05:26.809918 13662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:05:26 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:05:26.816075 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:26.801378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.2.86.62:40045)
0926 13:05:26.801580 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:26.801590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:26.801621 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:05:26.801746 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:26.801753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:26.801808 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:26.801946 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:05:26.802821 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:26.804225 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:26.805404 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:26.808419 (+ 3015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:26.811771 (+ 3352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:26.811910 (+ 139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:05:26.811914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:05:26.811914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:05:26.811923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:05:26.813951 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:26.814379 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:26.814381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:26.814382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:26.814482 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:26.814698 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:26.814699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:26.814699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:26.815032 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:05:26.815628 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:26.815701 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 13:05:26.815828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260926 13:05:26.818094 13662 catalog_manager.cc:1347] Generated new cluster ID: a91ea1eb93ac482790acd08b5199a59c
I20260926 13:05:26.818140 13662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:05:26.836637 13662 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:05:26.837527 13662 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:05:26.842487 13662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: Generated new TSK 0
I20260926 13:05:26.842926 13662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:05:27.600719 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:60111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:05:27.725572 13693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:05:27.725893 13693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:05:27.725941 13693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:05:27.727877 13693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:05:27.727949 13693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:05:27.727996 13693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:05:27.730448 13693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.86.20:60111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:05:27.730736 13693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:05:27.731066 13693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:05:27.736919 13699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:27.736994 13701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:05:27.737146 13693 server_base.cc:1034] running on GCE node
W20260926 13:05:27.737259 13698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:05:27.737720 13693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:05:27.738000 13693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:05:27.741303 13693 hybrid_clock.cc:648] HybridClock initialized: now 1790427927741275 us; error 40 us; skew 500 ppm
Sep 26 13:05:27 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:05:27.744187 13693 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:05:27.744748 13693 webserver.cc:460] Webserver started at http://127.2.86.1:36343/ using document root <none> and password file <none>
I20260926 13:05:27.745059 13693 fs_manager.cc:359] Metadata directory not provided
I20260926 13:05:27.745136 13693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:05:27.745292 13693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:05:27 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:27.758821 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "c753de8eb18a4de1b4979442d0b2a4a7"
format_stamp: "Formatted at 2026-09-26 13:05:27 on dist-test-slave-j4lq"
server_key: "4686bb13dfb524ec8c8a6089df565052"
server_key_iv: "e6b9ef9fe3d553d2cd06d57c265d13ed"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:27.759470 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "c753de8eb18a4de1b4979442d0b2a4a7"
format_stamp: "Formatted at 2026-09-26 13:05:27 on dist-test-slave-j4lq"
server_key: "4686bb13dfb524ec8c8a6089df565052"
server_key_iv: "e6b9ef9fe3d553d2cd06d57c265d13ed"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:27.765228 13693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260926 13:05:27.779286 13708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:27.779467 13693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:05:27.779556 13693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c753de8eb18a4de1b4979442d0b2a4a7"
format_stamp: "Formatted at 2026-09-26 13:05:27 on dist-test-slave-j4lq"
server_key: "4686bb13dfb524ec8c8a6089df565052"
server_key_iv: "e6b9ef9fe3d553d2cd06d57c265d13ed"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:27.779670 13693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:05:27.806751 13693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:05:27.809499 13693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:05:27.809719 13693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:05:27.810035 13693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:05:27.810469 13693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:05:27.810567 13693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:27.810668 13693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:05:27.810741 13693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:27.820088 13693 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:38155
I20260926 13:05:27.820668 13693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:05:27.821563 13823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:38155 every 8 connection(s)
Sep 26 13:05:27 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:05:27.830332 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 13693
I20260926 13:05:27.830425 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:05:27.830618 2392 external_mini_cluster.cc:1428] Setting key 6cac9139f59f0ec6a6a04aa3f57c7a78
I20260926 13:05:27.834830 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:27.822375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40045 (local address 127.2.86.1:58053)
0926 13:05:27.822824 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:27.822871 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:27.823499 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:27.823957 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:27.823962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:27.824219 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:05:27.824519 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:27.824526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:27.825622 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:27.825623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:27.826959 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:27.826962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:27.827547 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:27.827548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:27.827696 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:27.828388 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:05:27.828398 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:05:27.830840 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:05:27.833042 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:27.833051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:27.833054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:27.833377 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:27.833631 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:27.833632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:27.833633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:27.833780 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:27.834092 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:05:27.834098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:05:27.834288 (+ 190us) client_negotiation.cc:769] Sending connection context
0926 13:05:27.834402 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 13:05:27.834584 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":92,"threads_started":1}
I20260926 13:05:27.835408 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:27.822687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:58053 (local address 127.2.86.62:40045)
0926 13:05:27.823706 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:27.823708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:27.823717 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:05:27.823743 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:27.823744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:27.823756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:27.823822 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:05:27.824633 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:27.825510 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:27.827064 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:27.827376 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:27.827464 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:27.830973 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:05:27.830975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:05:27.830975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:05:27.830985 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:05:27.832938 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:27.833473 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:27.833485 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:27.833486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:27.833551 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:27.833859 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:27.833859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:27.833860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:27.834003 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:05:27.834065 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:27.835217 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0926 13:05:27.835310 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":981,"thread_start_us":68,"threads_started":1}
I20260926 13:05:27.836190 13824 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40045
I20260926 13:05:27.836342 13824 heartbeater.cc:464] Registering TS with master...
I20260926 13:05:27.836599 13824 heartbeater.cc:511] Master 127.2.86.62:40045 requested a full tablet report, sending...
I20260926 13:05:27.837216 13612 ts_manager.cc:194] Registered new tserver with Master: c753de8eb18a4de1b4979442d0b2a4a7 (127.2.86.1:38155)
I20260926 13:05:27.838130 13612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:58053
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:05:27.870023 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:60111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:05:27 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790427927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 13:05:28.000762 13830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:05:28.001109 13830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:05:28.001156 13830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:05:28.003115 13830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:05:28.003232 13830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:05:28.003294 13830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:05:28.006274 13830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.86.20:60111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:05:28.006780 13830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:05:28.007115 13830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:05:28.014070 13838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:28.014379 13837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:28.014590 13840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:05:28.015775 13830 server_base.cc:1034] running on GCE node
I20260926 13:05:28.016021 13830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:05:28.016345 13830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:05:28.017553 13830 hybrid_clock.cc:648] HybridClock initialized: now 1790427928017532 us; error 37 us; skew 500 ppm
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:05:28.020669 13830 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:05:28.021227 13830 webserver.cc:460] Webserver started at http://127.2.86.2:43267/ using document root <none> and password file <none>
I20260926 13:05:28.021487 13830 fs_manager.cc:359] Metadata directory not provided
I20260926 13:05:28.021543 13830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:05:28.021667 13830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:28.037034 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "3a21fc5a2e7f4fe7984a24ba811e4873"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "debd72daf93b15c55acdb57f4bed937f"
server_key_iv: "48a300644c63ba5f754d48c0b5e01ae7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.037525 13830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "3a21fc5a2e7f4fe7984a24ba811e4873"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "debd72daf93b15c55acdb57f4bed937f"
server_key_iv: "48a300644c63ba5f754d48c0b5e01ae7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.039397 13830 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:05:28.052008 13847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.052220 13830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:05:28.052326 13830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "3a21fc5a2e7f4fe7984a24ba811e4873"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "debd72daf93b15c55acdb57f4bed937f"
server_key_iv: "48a300644c63ba5f754d48c0b5e01ae7"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.052438 13830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:05:28.070003 13830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:05:28.072674 13830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:05:28.072911 13830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:05:28.073256 13830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:05:28.073685 13830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:05:28.073734 13830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.073777 13830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:05:28.073830 13830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.082547 13830 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:39249
I20260926 13:05:28.082978 13830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:05:28.083776 13961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:39249 every 8 connection(s)
I20260926 13:05:28.085234 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 13830
I20260926 13:05:28.085377 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:05:28.085579 2392 external_mini_cluster.cc:1428] Setting key f49758f0d3113fef70e79f5561c7b955
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
I20260926 13:05:28.096897 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.085863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:57457 (local address 127.2.86.62:40045)
0926 13:05:28.085913 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.085915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.085923 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:05:28.085946 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.085947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.085956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.086022 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:05:28.087067 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.087856 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.089569 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.089929 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.089975 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.093033 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:05:28.093035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:05:28.093036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:05:28.093045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:05:28.094985 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.095570 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.095572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.095573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.095636 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.095956 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.095957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.095957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.096118 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:05:28.096177 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.096644 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.096741 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:05:28.097080 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.083875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40045 (local address 127.2.86.2:57457)
0926 13:05:28.084049 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.084094 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.085013 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.086326 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.086332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.086637 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:05:28.086947 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.086955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.087997 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.087998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.089469 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.089472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.090071 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.090072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.090238 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.090896 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:05:28.090909 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:05:28.092860 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:05:28.095092 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:28.095100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:28.095103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:28.095454 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:28.095717 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:28.095718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:28.095718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:28.095867 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:28.096214 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:05:28.096220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:05:28.096443 (+ 223us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.096605 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.096772 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260926 13:05:28.097818 13962 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40045
I20260926 13:05:28.097963 13962 heartbeater.cc:464] Registering TS with master...
I20260926 13:05:28.098165 13962 heartbeater.cc:511] Master 127.2.86.62:40045 requested a full tablet report, sending...
I20260926 13:05:28.098613 13612 ts_manager.cc:194] Registered new tserver with Master: 3a21fc5a2e7f4fe7984a24ba811e4873 (127.2.86.2:39249)
I20260926 13:05:28.099133 13612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:57457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:05:28.124451 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:60111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:05:28.220738 13967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:05:28.221048 13967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:05:28.221092 13967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:05:28.222380 13967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:05:28.222462 13967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:05:28.222507 13967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:05:28.224225 13967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.86.20:60111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:05:28.224604 13967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:05:28.224908 13967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:05:28.229993 13972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:28.230525 13973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:05:28.230777 13975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:05:28.231810 13967 server_base.cc:1034] running on GCE node
I20260926 13:05:28.232159 13967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:05:28.232440 13967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:05:28.233635 13967 hybrid_clock.cc:648] HybridClock initialized: now 1790427928233611 us; error 39 us; skew 500 ppm
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:05:28.237797 13967 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:05:28.238512 13967 webserver.cc:460] Webserver started at http://127.2.86.3:39933/ using document root <none> and password file <none>
I20260926 13:05:28.239017 13967 fs_manager.cc:359] Metadata directory not provided
I20260926 13:05:28.239166 13967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:05:28.239375 13967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:28.257874 13967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "aac5a29498ee429a924827b032397538"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "053bd879681e87a99f5440f12bb800cc"
server_key_iv: "8237eb8c012465b9fd21f4f9da4386b0"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.258505 13967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "aac5a29498ee429a924827b032397538"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "053bd879681e87a99f5440f12bb800cc"
server_key_iv: "8237eb8c012465b9fd21f4f9da4386b0"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.263731 13967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 13:05:28.280827 13982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.281241 13967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:05:28.281361 13967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "aac5a29498ee429a924827b032397538"
format_stamp: "Formatted at 2026-09-26 13:05:28 on dist-test-slave-j4lq"
server_key: "053bd879681e87a99f5440f12bb800cc"
server_key_iv: "8237eb8c012465b9fd21f4f9da4386b0"
server_key_version: "kuduclusterkey@0"
I20260926 13:05:28.281463 13967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:05:28.305760 13967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:05:28.308003 13967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:05:28.308125 13967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:05:28.308444 13967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:05:28.308854 13967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:05:28.308935 13967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.308969 13967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:05:28.309018 13967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:05:28.315949 13967 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:34211
I20260926 13:05:28.316478 13967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:05:28.323494 14095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:34211 every 8 connection(s)
I20260926 13:05:28.324185 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 13967
I20260926 13:05:28.324280 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:05:28.324492 2392 external_mini_cluster.cc:1428] Setting key 2f11f2534234ad83b57e6adb01922ae6
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427928, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:05:28.333580 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.324593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:42199 (local address 127.2.86.62:40045)
0926 13:05:28.324644 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.324646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.324855 (+ 209us) server_negotiation.cc:407] Connection header received
0926 13:05:28.325622 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.325623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.325635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.325679 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:05:28.326261 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.326773 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.327798 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.328006 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.328096 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.330523 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:05:28.330524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:05:28.330525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:05:28.330533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:05:28.332035 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.332482 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.332484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.332484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.332547 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.332806 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.332807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.332807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.333020 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:05:28.333097 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.333401 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.333484 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:05:28.333900 14098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.324250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40045 (local address 127.2.86.3:42199)
0926 13:05:28.324722 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.324779 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.325512 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.325753 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.325757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.325958 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:05:28.326163 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.326167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.326876 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.326877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.327703 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.327705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.328107 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.328108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.328246 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.328817 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:05:28.328826 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:05:28.330419 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:05:28.332144 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:28.332152 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:28.332154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:28.332388 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:28.332614 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:05:28.332615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:05:28.332615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:05:28.332740 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:05:28.333100 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:05:28.333108 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:05:28.333269 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.333381 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.333546 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":97,"threads_started":1}
I20260926 13:05:28.334434 14096 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40045
I20260926 13:05:28.334561 14096 heartbeater.cc:464] Registering TS with master...
I20260926 13:05:28.334748 14096 heartbeater.cc:511] Master 127.2.86.62:40045 requested a full tablet report, sending...
I20260926 13:05:28.335160 13612 ts_manager.cc:194] Registered new tserver with Master: aac5a29498ee429a924827b032397538 (127.2.86.3:34211)
I20260926 13:05:28.335613 13612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:42199
I20260926 13:05:28.336272 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:05:28.346369 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.337886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.2.86.62:40045)
0926 13:05:28.337929 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.337931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.338223 (+ 292us) server_negotiation.cc:407] Connection header received
0926 13:05:28.338318 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.338319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.338329 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.338360 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:05:28.339173 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.339938 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.341343 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.341556 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.341636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.343219 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:05:28.343221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:05:28.343221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:05:28.343228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:05:28.345019 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.345433 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.345434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.345435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.345496 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:05:28.345708 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:05:28.345708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:05:28.345709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:05:28.345855 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:05:28.345935 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.346188 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.346270 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:05:28.348768 13612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:05:28.350550 13612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:05:28.358599 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34211 (local address 127.0.0.1:54884)
0926 13:05:28.353578 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.353623 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.353693 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.354354 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.354357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.354360 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:05:28.354440 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.354442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.355875 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.355876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.357321 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.357324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.358292 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.358293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.358368 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.358387 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.358436 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.358495 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":91,"threads_started":1}
I20260926 13:05:28.359119 14109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38155 (local address 127.0.0.1:38476)
0926 13:05:28.354553 (+ 1338us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.354564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.354611 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.354849 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.354850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.354851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:05:28.354888 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.354889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.356508 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.356509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.357589 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.357591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.358861 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.358862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.358923 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.358927 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.358976 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.359022 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260926 13:05:28.359392 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.2.86.3:34211)
0926 13:05:28.354168 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.354173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.354183 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:05:28.354225 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.354227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.354245 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.354526 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:05:28.354603 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.355780 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.357438 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.358190 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.359156 (+ 966us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.359253 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.359279 (+ 26us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.359318 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260926 13:05:28.359582 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.2.86.1:38155)
0926 13:05:28.354460 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.354465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.354668 (+ 203us) server_negotiation.cc:407] Connection header received
0926 13:05:28.354703 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.354705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.354720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.354785 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:05:28.355038 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.356414 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.357674 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.358703 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.358770 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.358831 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.359440 (+ 609us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.359505 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":70,"threads_started":1}
I20260926 13:05:28.360208 14020 tablet_service.cc:1467] Processing CreateTablet for tablet 057fe2ec349343eda79e5716ec387e2e (DEFAULT_TABLE table=test-table [id=4ba85ae6634a436e81771bffdfee3b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:05:28.360349 13758 tablet_service.cc:1467] Processing CreateTablet for tablet 057fe2ec349343eda79e5716ec387e2e (DEFAULT_TABLE table=test-table [id=4ba85ae6634a436e81771bffdfee3b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:05:28.360579 14020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057fe2ec349343eda79e5716ec387e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:05:28.360608 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057fe2ec349343eda79e5716ec387e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:05:28.364827 14115 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Bootstrap starting.
I20260926 13:05:28.365907 14115 tablet_bootstrap.cc:654] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Neither blocks nor log segments found. Creating new log.
I20260926 13:05:28.366130 14114 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: Bootstrap starting.
I20260926 13:05:28.366835 14114 tablet_bootstrap.cc:654] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: Neither blocks nor log segments found. Creating new log.
I20260926 13:05:28.367228 14115 log.cc:824] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Log is configured to *not* fsync() on all Append() calls
I20260926 13:05:28.368023 14114 log.cc:824] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:05:28.368719 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:39249 (local address 127.0.0.1:35682)
0926 13:05:28.359038 (+ 5592us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.359048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.359079 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.364636 (+ 5557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.364637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.364639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:05:28.364684 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.364686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.366071 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.366072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.367200 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.367204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.368472 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.368473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.368555 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.368561 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.368594 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.368646 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5582,"thread_start_us":75,"threads_started":1}
I20260926 13:05:28.368918 14115 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: No bootstrap required, opened a new log
I20260926 13:05:28.368991 14115 ts_tablet_manager.cc:1397] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260926 13:05:28.369387 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.353958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.2.86.2:39249)
0926 13:05:28.364462 (+ 10504us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.364466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.364477 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:05:28.364511 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.364512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.364526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.364638 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:05:28.364768 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.365962 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.367338 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.368368 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.369155 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.369240 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.369269 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.369310 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10468,"thread_start_us":93,"threads_started":1}
I20260926 13:05:28.370187 13888 tablet_service.cc:1467] Processing CreateTablet for tablet 057fe2ec349343eda79e5716ec387e2e (DEFAULT_TABLE table=test-table [id=4ba85ae6634a436e81771bffdfee3b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:05:28.370960 13888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057fe2ec349343eda79e5716ec387e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:05:28.371508 14115 raft_consensus.cc:348] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.371816 14115 raft_consensus.cc:374] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:05:28.371871 14115 raft_consensus.cc:729] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac5a29498ee429a924827b032397538, State: Initialized, Role: FOLLOWER
I20260926 13:05:28.372027 14115 consensus_queue.cc:260] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.372311 14115 ts_tablet_manager.cc:1428] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 13:05:28.372974 14096 heartbeater.cc:503] Master 127.2.86.62:40045 was elected leader, sending a full tablet report...
I20260926 13:05:28.373054 14114 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: No bootstrap required, opened a new log
I20260926 13:05:28.373118 14114 ts_tablet_manager.cc:1397] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260926 13:05:28.375883 14114 raft_consensus.cc:348] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.376145 14114 raft_consensus.cc:374] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:05:28.376204 14114 raft_consensus.cc:729] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c753de8eb18a4de1b4979442d0b2a4a7, State: Initialized, Role: FOLLOWER
I20260926 13:05:28.376336 14114 consensus_queue.cc:260] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.376639 13824 heartbeater.cc:503] Master 127.2.86.62:40045 was elected leader, sending a full tablet report...
I20260926 13:05:28.376916 14114 ts_tablet_manager.cc:1428] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 13:05:28.377792 14119 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: Bootstrap starting.
I20260926 13:05:28.378811 14119 tablet_bootstrap.cc:654] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: Neither blocks nor log segments found. Creating new log.
I20260926 13:05:28.379211 14119 log.cc:824] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: Log is configured to *not* fsync() on all Append() calls
I20260926 13:05:28.380074 14119 tablet_bootstrap.cc:492] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: No bootstrap required, opened a new log
I20260926 13:05:28.380141 14119 ts_tablet_manager.cc:1397] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:05:28.382052 14119 raft_consensus.cc:348] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.382181 14119 raft_consensus.cc:374] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:05:28.382202 14119 raft_consensus.cc:729] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a21fc5a2e7f4fe7984a24ba811e4873, State: Initialized, Role: FOLLOWER
I20260926 13:05:28.382329 14119 consensus_queue.cc:260] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.382606 14119 ts_tablet_manager.cc:1428] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:05:28.382805 13962 heartbeater.cc:503] Master 127.2.86.62:40045 was elected leader, sending a full tablet report...
Sep 26 13:05:28 dist-test-slave-j4lq krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:05:28.492911 14122 raft_consensus.cc:482] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:05:28.493070 14122 raft_consensus.cc:504] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.493498 14122 leader_election.cc:288] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aac5a29498ee429a924827b032397538 (127.2.86.3:34211), c753de8eb18a4de1b4979442d0b2a4a7 (127.2.86.1:38155)
I20260926 13:05:28.497800 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.493763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34211 (local address 127.2.86.2:32843)
0926 13:05:28.494083 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.494096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.494146 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.494478 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.494479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.494482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:05:28.494555 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.494557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.495498 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.495499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.496698 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.496701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.497538 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.497539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.497619 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.497622 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.497673 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.497721 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
I20260926 13:05:28.498368 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.494265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:32843 (local address 127.2.86.3:34211)
0926 13:05:28.494310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.494311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.494320 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:05:28.494352 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.494353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.494366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.494421 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:05:28.494634 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.495351 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.496816 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.497411 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.497488 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.497502 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.498237 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.498299 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:05:28.498654 14038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057fe2ec349343eda79e5716ec387e2e" candidate_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aac5a29498ee429a924827b032397538" is_pre_election: true
I20260926 13:05:28.498821 14038 raft_consensus.cc:2454] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a21fc5a2e7f4fe7984a24ba811e4873 in term 0.
I20260926 13:05:28.499176 13848 leader_election.cc:302] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a21fc5a2e7f4fe7984a24ba811e4873, aac5a29498ee429a924827b032397538; no voters:
I20260926 13:05:28.499316 14122 raft_consensus.cc:2781] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:05:28.499357 14122 raft_consensus.cc:482] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:05:28.499382 14122 raft_consensus.cc:3037] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:05:28.500290 14122 raft_consensus.cc:504] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.500434 14122 leader_election.cc:288] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [CANDIDATE]: Term 1 election: Requested vote from peers aac5a29498ee429a924827b032397538 (127.2.86.3:34211), c753de8eb18a4de1b4979442d0b2a4a7 (127.2.86.1:38155)
I20260926 13:05:28.500948 14038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057fe2ec349343eda79e5716ec387e2e" candidate_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aac5a29498ee429a924827b032397538"
I20260926 13:05:28.501044 14038 raft_consensus.cc:3037] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:05:28.501931 14038 raft_consensus.cc:2454] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a21fc5a2e7f4fe7984a24ba811e4873 in term 1.
I20260926 13:05:28.502200 13848 leader_election.cc:302] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a21fc5a2e7f4fe7984a24ba811e4873, aac5a29498ee429a924827b032397538; no voters:
I20260926 13:05:28.502327 14122 raft_consensus.cc:2781] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:05:28.502490 14122 raft_consensus.cc:686] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [term 1 LEADER]: Becoming Leader. State: Replica: 3a21fc5a2e7f4fe7984a24ba811e4873, State: Running, Role: LEADER
I20260926 13:05:28.502588 14122 consensus_queue.cc:237] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } }
I20260926 13:05:28.503505 13611 catalog_manager.cc:5495] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a21fc5a2e7f4fe7984a24ba811e4873 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 39249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260926 13:05:28.510730 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.493946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35431 (local address 127.2.86.1:38155)
0926 13:05:28.494219 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.494222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.494469 (+ 247us) server_negotiation.cc:407] Connection header received
0926 13:05:28.494602 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.494603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.494618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.494650 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:05:28.494886 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.495719 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.509750 (+ 14031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.510279 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.510354 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.510376 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.510603 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.510653 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260926 13:05:28.510743 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.493878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38155 (local address 127.2.86.2:35431)
0926 13:05:28.494370 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.494385 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:05:28.494503 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:05:28.494722 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:05:28.494723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:05:28.494724 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:05:28.494806 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.494808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.495829 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.495830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.509621 (+ 13791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:05:28.509628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.510403 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.510405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:05:28.510493 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.510498 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:05:28.510575 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:05:28.510625 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":117,"threads_started":1}
I20260926 13:05:28.511168 13762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057fe2ec349343eda79e5716ec387e2e" candidate_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" is_pre_election: true
I20260926 13:05:28.511344 13762 raft_consensus.cc:2454] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a21fc5a2e7f4fe7984a24ba811e4873 in term 0.
I20260926 13:05:28.511499 13775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057fe2ec349343eda79e5716ec387e2e" candidate_uuid: "3a21fc5a2e7f4fe7984a24ba811e4873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c753de8eb18a4de1b4979442d0b2a4a7"
I20260926 13:05:28.511572 13775 raft_consensus.cc:3037] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:05:28.512508 13775 raft_consensus.cc:2454] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a21fc5a2e7f4fe7984a24ba811e4873 in term 1.
I20260926 13:05:28.547339 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.542736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.2.86.2:39249)
0926 13:05:28.542784 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:05:28.542786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:05:28.542810 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:05:28.542975 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:05:28.542976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:05:28.543029 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:05:28.543123 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:05:28.543321 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.544260 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.546034 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:05:28.546441 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:05:28.546507 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:05:28.546746 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:05:28.546894 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:05:28.546973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 13:05:28.547226 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 13:05:28.547249 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:05:28.549595 13775 raft_consensus.cc:1261] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 1 FOLLOWER]: Refusing update from remote peer 3a21fc5a2e7f4fe7984a24ba811e4873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:05:28.549599 14038 raft_consensus.cc:1261] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 1 FOLLOWER]: Refusing update from remote peer 3a21fc5a2e7f4fe7984a24ba811e4873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:05:28.550009 14124 consensus_queue.cc:1035] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c753de8eb18a4de1b4979442d0b2a4a7" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:05:28.550424 14124 consensus_queue.cc:1035] T 057fe2ec349343eda79e5716ec387e2e P 3a21fc5a2e7f4fe7984a24ba811e4873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aac5a29498ee429a924827b032397538" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:05:28.553843 14127 mvcc.cc:205] Tried to move back new op lower bound from 7333592795335925760 to 7333592795147124736. Current Snapshot: MvccSnapshot[applied={T|T < 7333592795335925760}]
I20260926 13:05:28.554909 14129 mvcc.cc:205] Tried to move back new op lower bound from 7333592795335925760 to 7333592795147124736. Current Snapshot: MvccSnapshot[applied={T|T < 7333592795335925760}]
I20260926 13:05:28.555495 14128 mvcc.cc:205] Tried to move back new op lower bound from 7333592795335925760 to 7333592795147124736. Current Snapshot: MvccSnapshot[applied={T|T < 7333592795335925760}]
I20260926 13:05:28.560348 13611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:05:28.560485 13611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:05:28.561457 13611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8111885e336c489887eea0c9a2419aca: Sending DeleteTablet for 3 replicas of tablet 057fe2ec349343eda79e5716ec387e2e
I20260926 13:05:28.562053 14020 tablet_service.cc:1514] Processing DeleteTablet for tablet 057fe2ec349343eda79e5716ec387e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:05:28 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:54884
I20260926 13:05:28.562155 13758 tablet_service.cc:1514] Processing DeleteTablet for tablet 057fe2ec349343eda79e5716ec387e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:05:28 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:38476
I20260926 13:05:28.562405 14137 tablet_replica.cc:330] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7: stopping tablet replica
I20260926 13:05:28.562428 13888 tablet_service.cc:1514] Processing DeleteTablet for tablet 057fe2ec349343eda79e5716ec387e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:05:28 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:35682
I20260926 13:05:28.562556 14137 raft_consensus.cc:2229] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:05:28.562675 14137 raft_consensus.cc:2258] T 057fe2ec349343eda79e5716ec387e2e P c753de8eb18a4de1b4979442d0b2a4a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:05:28.563012 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 13693
W20260926 13:05:28.571559 13598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:38155 (error 108)
I20260926 13:05:28.571794 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 13830
I20260926 13:05:28.572605 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.572026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38155 (local address 127.0.0.1:38490)
0926 13:05:28.572514 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.572559 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464}
W20260926 13:05:28.573077 13598 catalog_manager.cc:4584] TS c753de8eb18a4de1b4979442d0b2a4a7 (127.2.86.1:38155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 057fe2ec349343eda79e5716ec387e2e: Network error: Client connection negotiation failed: client connection to 127.2.86.1:38155: connect: Connection refused (error 111)
I20260926 13:05:28.571825 14136 tablet_replica.cc:330] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: stopping tablet replica
I20260926 13:05:28.573377 14136 raft_consensus.cc:2229] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:05:28.573565 14136 raft_consensus.cc:2258] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:05:28.574164 14136 ts_tablet_manager.cc:1905] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:05:28.577073 14136 ts_tablet_manager.cc:1918] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:05:28.577171 14136 log.cc:1196] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/057fe2ec349343eda79e5716ec387e2e
I20260926 13:05:28.577833 14136 ts_tablet_manager.cc:1939] T 057fe2ec349343eda79e5716ec387e2e P aac5a29498ee429a924827b032397538: Deleting consensus metadata
I20260926 13:05:28.578444 13596 catalog_manager.cc:4841] TS aac5a29498ee429a924827b032397538 (127.2.86.3:34211): tablet 057fe2ec349343eda79e5716ec387e2e (table test-table [id=4ba85ae6634a436e81771bffdfee3b73]) successfully deleted
I20260926 13:05:28.579797 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:05:28.579681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:39249 (local address 127.0.0.1:35710)
0926 13:05:28.579735 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 13:05:28.579760 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.2:39249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:05:28.580143 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 13967
I20260926 13:05:28.587447 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 13580
I20260926 13:05:28.595224 2392 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 13:05:28.940594 2392 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 13:05:28.940711 2392 mini_ranger.cc:66] Stopping Ranger...
I20260926 13:05:29.285601 2392 mini_ranger.cc:68] Stopped Ranger
2026-09-26 13:05:29.285 UTC [13166] LOG: received smart shutdown request
2026-09-26 13:05:29.289 UTC [13166] LOG: background worker "logical replication launcher" (PID 13175) exited with exit code 1
2026-09-26 13:05:29.289 UTC [13170] LOG: shutting down
2026-09-26 13:05:29.479 UTC [13166] LOG: database system is shut down
2026-09-26T13:05:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45932 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:05:29 dist-test-slave-j4lq krb5kdc[14148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:05:29 dist-test-slave-j4lq krb5kdc[14148](info): set up 2 sockets
Sep 26 13:05:29 dist-test-slave-j4lq krb5kdc[14148](info): commencing operation
krb5kdc: starting...
W20260926 13:05:30.636991 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:05:30 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:05:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:05:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangeradmin_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangeradmin_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangerlookup_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangerlookup_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
I20260926 13:05:30.812602 2392 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNbmtbn/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/postgres -l logfile start
2026-09-26 13:05:33.446 UTC [14178] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 13:05:33.446 UTC [14178] LOG: listening on IPv4 address "127.2.86.20", port 59959
2026-09-26 13:05:33.450 UTC [14178] LOG: listening on Unix socket "/tmp/.s.PGSQL.59959"
2026-09-26 13:05:33.472 UTC [14181] LOG: database system was shut down at 2026-09-26 13:05:31 UTC
2026-09-26 13:05:33.476 UTC [14178] LOG: database system is ready to accept connections
I20260926 13:05:34.412808 2392 mini_postgres.cc:96] Postgres bound to 59959
2026-09-26 13:05:34.418 UTC [14189] FATAL: database "slave" does not exist
127.2.86.20:59959 - accepting connections
I20260926 13:05:34.419180 2392 mini_ranger.cc:161] Starting Ranger...
I20260926 13:05:34.433452 2392 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 13:05:34.963937 2392 mini_ranger.cc:87] Created ranger Postgres database
I20260926 13:05:34.964041 2392 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin
2026-09-26 13:05:35,247 [I] DB FLAVOR :POSTGRES
2026-09-26 13:05:35,248 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:05:35,248 [I] Checking connection..
2026-09-26 13:05:35,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 13:05:35,495 [I] Checking connection passed.
2026-09-26 13:05:35,495 [I] --------- Verifying version history table ---------
2026-09-26 13:05:35,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:05:35,725 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:05:35,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:05:35,958 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 13:05:35,958 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 13:05:35,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 13:05:36.180 UTC [14253] WARNING: there is no transaction in progress
2026-09-26 13:05:36,186 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 13:05:36,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 13:05:36,415 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 13:05:36,416 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:05:36,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 13:05:36,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 13:05:36,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j4lq.c.gcp-upstream.internal','N') ;"
2026-09-26 13:05:37,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:05:37,423 [I] Table x_portal_user does not exist in database ranger
2026-09-26 13:05:37,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:05:37,657 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 13:05:37,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:05:37,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:05:38,159 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 13:05:38,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 13:05:39.146 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.162 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.181 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.196 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.212 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.231 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.286 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.293 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.308 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.319 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.330 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.345 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.355 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.362 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.370 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:39.377 UTC [14361] WARNING: there is no transaction in progress
2026-09-26 13:05:40,096 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 13:05:40,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 13:05:40,340 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 13:05:40,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 13:05:40,605 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 13:05:40,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 13:05:40,844 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 13:05:40,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 13:05:41,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 13:05:41,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j4lq.c.gcp-upstream.internal';"
2026-09-26 13:05:41,576 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 13:05:41,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 13:05:41,823 [I] DB_PATCHES have already been applied
I20260926 13:05:41.829005 2392 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNbmtbn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 13:05:41.829161 2392 mini_ranger.cc:193] Using host: 127.2.86.20
I20260926 13:05:41.832162 2392 mini_ranger.cc:227] Ranger admin URL: http://127.2.86.20:33057
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 13:05:42.990785 2392 mini_ranger.cc:160] Time spent starting Ranger: real 8.572s user 0.000s sys 0.009s
Sep 26, 2026 1:06:03 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 1:06:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 1:06:03 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 1:06:03 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 1:06:04 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 13:06:05.158404 2392 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.86.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.86.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangerkms_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.86.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/rangerkms_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.86.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.86.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.86.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/HTTP_127.2.86.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 13:06:05 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427965, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 13:06:05.248070 2392 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 13:06:05.257705 2392 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 13:06:06.246817 2392 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 13:06:06.246902 2392 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms
I20260926 13:06:06.246925 2392 mini_ranger_kms.cc:227] Using postgres at 127.2.86.20:59959
2026-09-26 13:06:06,412 [I] DB FLAVOR :POSTGRES
2026-09-26 13:06:06,412 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 13:06:06,412 [I] Checking connection
2026-09-26 13:06:06,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 13:06:06,662 [I] connection success
2026-09-26 13:06:06,663 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 13:06:06,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 13:06:06,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 13:06:06,904 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 13:06:06,904 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 13:06:06,905 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 13:06:06,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.86.20:59959/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 13:06:07,186 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 13:06:07.655051 2392 mini_ranger_kms.cc:310] Created kms service
I20260926 13:06:07.843776 2392 mini_ranger_kms.cc:326] Created kudu user
I20260926 13:06:07.887866 2392 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 13:06:07 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427965, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:08.287122 2392 mini_ranger_kms.cc:384] Added ranger policy
I20260926 13:06:08.287243 2392 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNbmtbn/build/release/bin/postgresql.jar:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNbmtbn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 13:06:08.287262 2392 mini_ranger_kms.cc:240] Using host: 127.2.86.20
I20260926 13:06:08.291041 2392 mini_ranger_kms.cc:276] Ranger KMS PID: 14541
I20260926 13:06:08.291137 2392 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.86.20:35169
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 1:06:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 13:06:10.495288 2392 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.247s user 0.002s sys 0.147s
I20260926 13:06:10.495364 2392 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 13:06:10.495399 2392 mini_ranger_kms.cc:401] 127.2.86.20:35169/kms/v1/keys
Sep 26 13:06:10 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790427970, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 13:06:11 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427970, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
Sep 26 13:06:11 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427970, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.86.20@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
Sep 26 13:06:13 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:06:13.752352 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40823
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40823
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:13.833853 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:13.834090 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:13.834129 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:13.835269 14592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:13.835335 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:13.835361 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:13.835391 14592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:13.835415 14592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:13.837101 14592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.86.20:35169/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40823
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:13.837363 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:13.837580 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:13.842463 14597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:13.842590 14592 server_base.cc:1034] running on GCE node
W20260926 13:06:13.842427 14598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:13.842427 14600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:13.842957 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:13.843286 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:13.844439 14592 hybrid_clock.cc:648] HybridClock initialized: now 1790427973844425 us; error 32 us; skew 500 ppm
Sep 26 13:06:13 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427973, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:13.853354 14592 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:13.853925 14592 webserver.cc:460] Webserver started at http://127.2.86.62:39433/ using document root <none> and password file <none>
I20260926 13:06:13.854193 14592 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:13.854247 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:13.854374 14592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:06:13 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427973, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:13.903578 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "d4df3911fe5045f2ac616303909c1521"
format_stamp: "Formatted at 2026-09-26 13:06:13 on dist-test-slave-j4lq"
server_key: "b27701784aeb77600b1df80160c46fbe"
server_key_iv: "5d49163ee1f28898e679467019d89948"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:13.904007 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "d4df3911fe5045f2ac616303909c1521"
format_stamp: "Formatted at 2026-09-26 13:06:13 on dist-test-slave-j4lq"
server_key: "b27701784aeb77600b1df80160c46fbe"
server_key_iv: "5d49163ee1f28898e679467019d89948"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:13.905535 14592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:06:13.916854 14607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:13.917119 14592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:13.917217 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "d4df3911fe5045f2ac616303909c1521"
format_stamp: "Formatted at 2026-09-26 13:06:13 on dist-test-slave-j4lq"
server_key: "b27701784aeb77600b1df80160c46fbe"
server_key_iv: "5d49163ee1f28898e679467019d89948"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:13.917313 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:13.934297 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:13.936369 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:13.936543 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:13.941692 14592 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40823
I20260926 13:06:13.941777 14659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40823 every 8 connection(s)
I20260926 13:06:13.942204 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:13.942986 14660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:13.946265 14660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Bootstrap starting.
I20260926 13:06:13.947024 14660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:13.947362 14660 log.cc:824] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:13.948204 14660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: No bootstrap required, opened a new log
I20260926 13:06:13.948444 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 14592
I20260926 13:06:13.948565 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:06:13.948799 2392 external_mini_cluster.cc:1428] Setting key 985d2b5260c15d4a2137d22b4aee4594
I20260926 13:06:13.951220 14660 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:13.951468 14660 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:13.951540 14660 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4df3911fe5045f2ac616303909c1521, State: Initialized, Role: FOLLOWER
I20260926 13:06:13.951736 14660 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:13.951848 14660 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:13.951906 14660 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:13.951988 14660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:13.952826 14660 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:13.952970 14660 leader_election.cc:302] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4df3911fe5045f2ac616303909c1521; no voters:
I20260926 13:06:13.953159 14660 leader_election.cc:288] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:06:13.953297 14665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:13.953459 14660 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:13.953465 14665 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 LEADER]: Becoming Leader. State: Replica: d4df3911fe5045f2ac616303909c1521, State: Running, Role: LEADER
I20260926 13:06:13.954664 14665 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:13.955673 14666 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4df3911fe5045f2ac616303909c1521. Latest consensus state: current_term: 1 leader_uuid: "d4df3911fe5045f2ac616303909c1521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } } }
I20260926 13:06:13.955766 14666 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:13.956046 14667 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4df3911fe5045f2ac616303909c1521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } } }
I20260926 13:06:13.956131 14667 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:13.956248 14680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:13.956800 14680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:06:13 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:13.958403 14660 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:06:13.962139 14680 catalog_manager.cc:1347] Generated new cluster ID: cc4d4acc41274977bf5019aa34753e04
I20260926 13:06:13.962242 14680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:06:13.967594 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:13.949463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.2.86.62:40823)
0926 13:06:13.949643 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:13.949653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:13.949737 (+ 84us) server_negotiation.cc:407] Connection header received
0926 13:06:13.951210 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:13.951219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:13.951288 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:13.951650 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:13.952294 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:13.953552 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:13.954569 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:13.955224 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:13.955350 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:13.958346 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:13.958350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:13.958351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:13.958364 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:13.965034 (+ 6670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:13.965512 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:13.965516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:13.965516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:13.965641 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:13.965827 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:13.965827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:13.965828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:13.966230 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:13.966775 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:13.966869 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:06:13.967295 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260926 13:06:13.989380 14680 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:06:13.990267 14680 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:13.994681 14680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Generated new TSK 0
I20260926 13:06:13.994885 14680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:06:14.728765 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:14.833149 14705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:14.833364 14705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:14.833388 14705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:14.834686 14705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:14.834756 14705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:14.834800 14705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:14.837112 14705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:14.837380 14705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:14.837644 14705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:14.843686 14712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:14.843977 14714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:14.844099 14711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:14.845249 14705 server_base.cc:1034] running on GCE node
I20260926 13:06:14.845476 14705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:14.845743 14705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:14.846918 14705 hybrid_clock.cc:648] HybridClock initialized: now 1790427974846897 us; error 35 us; skew 500 ppm
Sep 26 13:06:14 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427974, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:14.850184 14705 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:14.850737 14705 webserver.cc:460] Webserver started at http://127.2.86.1:45857/ using document root <none> and password file <none>
I20260926 13:06:14.851011 14705 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:14.851083 14705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:14.851269 14705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:06:14 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427974, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:14.877836 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "21ae7ba2550248aa92eb2adecd8e6127"
format_stamp: "Formatted at 2026-09-26 13:06:14 on dist-test-slave-j4lq"
server_key: "1565de6b4747a66db817c7f2a4683b95"
server_key_iv: "7ca7f9d822514c4bd143d37c8c4a3d21"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:14.878322 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "21ae7ba2550248aa92eb2adecd8e6127"
format_stamp: "Formatted at 2026-09-26 13:06:14 on dist-test-slave-j4lq"
server_key: "1565de6b4747a66db817c7f2a4683b95"
server_key_iv: "7ca7f9d822514c4bd143d37c8c4a3d21"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:14.881264 14705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 13:06:14.894585 14722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:14.894806 14705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:14.894903 14705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "21ae7ba2550248aa92eb2adecd8e6127"
format_stamp: "Formatted at 2026-09-26 13:06:14 on dist-test-slave-j4lq"
server_key: "1565de6b4747a66db817c7f2a4683b95"
server_key_iv: "7ca7f9d822514c4bd143d37c8c4a3d21"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:14.894990 14705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:14.928658 14705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:14.931296 14705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:14.931661 14705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:14.931993 14705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:14.932526 14705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:14.932643 14705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:14.932746 14705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:14.932832 14705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:14.939287 14705 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:42699
I20260926 13:06:14.939797 14705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:06:14.943491 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 14705
I20260926 13:06:14.943595 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:06:14.943795 2392 external_mini_cluster.cc:1428] Setting key 3f4ff4416d6d8c47923dedd88e4211bf
I20260926 13:06:14.945266 14835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:42699 every 8 connection(s)
Sep 26 13:06:14 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427974, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
I20260926 13:06:14.956856 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:14.946289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:59583 (local address 127.2.86.62:40823)
0926 13:06:14.947425 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:14.947427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:14.947435 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:06:14.947458 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:14.947459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:14.947471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:14.947546 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:14.948324 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:14.949148 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:14.950608 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:14.950905 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:14.950979 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:14.953258 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:14.953260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:14.953260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:14.953268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:14.955165 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:14.955716 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:14.955718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:14.955718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:14.955786 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:14.956061 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:14.956062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:14.956063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:14.956186 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:14.956257 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:14.956572 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 13:06:14.956656 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":77,"threads_started":1}
I20260926 13:06:14.956966 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:14.946004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.1:59583)
0926 13:06:14.946557 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:14.946616 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:14.947241 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:14.947655 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:14.947660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:14.947914 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:14.948199 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:14.948207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:14.949260 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:14.949261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:14.950504 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:14.950507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:14.951075 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:14.951076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:14.951219 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:14.951854 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:14.951862 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:14.953154 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:14.955256 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:14.955262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:14.955265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:14.955617 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:14.955855 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:14.955855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:14.955856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:14.955983 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:14.956262 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:14.956268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:14.956437 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 13:06:14.956552 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 13:06:14.956708 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
I20260926 13:06:14.957798 14836 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:14.957908 14836 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:14.958108 14836 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:14.958689 14615 ts_manager.cc:194] Registered new tserver with Master: 21ae7ba2550248aa92eb2adecd8e6127 (127.2.86.1:42699)
I20260926 13:06:14.959576 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:59583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:06:14.976338 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 13:06:15.089710 14844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:15.090010 14844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:15.090054 14844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:15.091260 14844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:15.091332 14844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:15.091378 14844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:15.092978 14844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:15.093228 14844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:15.093462 14844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:15.099920 14850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.100239 14849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.099932 14852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:15.100337 14844 server_base.cc:1034] running on GCE node
I20260926 13:06:15.100550 14844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:15.100849 14844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:15.102133 14844 hybrid_clock.cc:648] HybridClock initialized: now 1790427975102132 us; error 54 us; skew 500 ppm
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:15.105521 14844 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:15.106076 14844 webserver.cc:460] Webserver started at http://127.2.86.2:46813/ using document root <none> and password file <none>
I20260926 13:06:15.106343 14844 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:15.106405 14844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:15.106600 14844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:15.121395 14844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "a639a56e2913427b9bb90814a43dec68"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "f6d0b101ee9c0e53c6e44ec9c2f54fa5"
server_key_iv: "3f63bf66e30a4310aeddcb3b7611110f"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.122226 14844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "a639a56e2913427b9bb90814a43dec68"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "f6d0b101ee9c0e53c6e44ec9c2f54fa5"
server_key_iv: "3f63bf66e30a4310aeddcb3b7611110f"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.125267 14844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:06:15.145063 14860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.145325 14844 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260926 13:06:15.145426 14844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "a639a56e2913427b9bb90814a43dec68"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "f6d0b101ee9c0e53c6e44ec9c2f54fa5"
server_key_iv: "3f63bf66e30a4310aeddcb3b7611110f"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.145526 14844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:15.167680 14844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:15.170110 14844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:15.170325 14844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:15.170656 14844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:15.171098 14844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:15.171190 14844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.171272 14844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:15.171334 14844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.178262 14844 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:36751
I20260926 13:06:15.178763 14844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:06:15.181950 14973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:36751 every 8 connection(s)
I20260926 13:06:15.187096 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 14844
I20260926 13:06:15.187206 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:06:15.187409 2392 external_mini_cluster.cc:1428] Setting key dcfa9b2bc4b62479ecce64e3e8df658f
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
I20260926 13:06:15.210898 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.182749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.2:60251)
0926 13:06:15.187751 (+ 5002us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:15.187816 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:15.190221 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:15.190628 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:15.190633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:15.190924 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:15.191222 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.191229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.192286 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.192287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.193597 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.193600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.194190 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.194191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.194336 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.205055 (+ 10719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:15.205070 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:15.206887 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:15.209115 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.209127 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.209130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.209477 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.209703 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.209703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.209704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.209857 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.210165 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:15.210171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:15.210369 (+ 198us) client_negotiation.cc:769] Sending connection context
0926 13:06:15.210490 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 13:06:15.210655 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4941,"thread_start_us":101,"threads_started":1}
I20260926 13:06:15.211798 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.182691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:60251 (local address 127.2.86.62:40823)
0926 13:06:15.182748 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:15.182750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:15.187939 (+ 5189us) server_negotiation.cc:407] Connection header received
0926 13:06:15.190370 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:15.190371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:15.190385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:15.190417 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:15.191340 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.192162 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.193708 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.194030 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.194079 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.207058 (+ 12979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:15.207061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:15.207061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:15.207071 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:15.208997 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.209571 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.209573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.209573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.209635 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.209930 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.209931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.209931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.210079 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:15.210139 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:15.211584 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0926 13:06:15.211684 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 13:06:15.212436 14974 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:15.212536 14974 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:15.212769 14974 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:15.213160 14615 ts_manager.cc:194] Registered new tserver with Master: a639a56e2913427b9bb90814a43dec68 (127.2.86.2:36751)
I20260926 13:06:15.213676 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:60251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:06:15.233023 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:15.344738 14979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:15.345084 14979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:15.345168 14979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:15.346381 14979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:15.346457 14979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:15.346503 14979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:15.348047 14979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:15.348323 14979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:15.348592 14979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:15.355476 14987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.355805 14985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.356335 14984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:15.357198 14979 server_base.cc:1034] running on GCE node
I20260926 13:06:15.357430 14979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:15.357702 14979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:15.373096 14979 hybrid_clock.cc:648] HybridClock initialized: now 1790427975373047 us; error 59 us; skew 500 ppm
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:15.376942 14979 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:15.377564 14979 webserver.cc:460] Webserver started at http://127.2.86.3:41805/ using document root <none> and password file <none>
I20260926 13:06:15.377861 14979 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:15.377933 14979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:15.378073 14979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:15.394229 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "a3d9d57ad0924ae284734bd25aef8ee7"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "3f263fc21bf376d757c7b96a95aa85fc"
server_key_iv: "61689925279a36cafe52bf5c603e7bf9"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.394706 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "a3d9d57ad0924ae284734bd25aef8ee7"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "3f263fc21bf376d757c7b96a95aa85fc"
server_key_iv: "61689925279a36cafe52bf5c603e7bf9"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.400550 14979 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260926 13:06:15.421321 14994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.422478 14979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 13:06:15.422668 14979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a3d9d57ad0924ae284734bd25aef8ee7"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "3f263fc21bf376d757c7b96a95aa85fc"
server_key_iv: "61689925279a36cafe52bf5c603e7bf9"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.422854 14979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:15.444008 14979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:15.446702 14979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:15.446929 14979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:15.447253 14979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:15.447667 14979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:15.447765 14979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.447839 14979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:15.447896 14979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.458806 14979 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:33587
I20260926 13:06:15.459342 14979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:06:15.460427 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 14979
I20260926 13:06:15.460533 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:06:15.460736 2392 external_mini_cluster.cc:1428] Setting key 150c15e831d95cfd7ded9340bf80afd6
I20260926 13:06:15.462914 15107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:33587 every 8 connection(s)
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:15.476094 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.464070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:48615 (local address 127.2.86.62:40823)
0926 13:06:15.464113 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:15.464115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:15.464448 (+ 333us) server_negotiation.cc:407] Connection header received
0926 13:06:15.465185 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:15.465186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:15.465199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:15.465231 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:15.466122 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.467039 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.468701 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.469009 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.469062 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.471884 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:15.471886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:15.471886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:15.471894 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:15.474378 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.474923 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.474925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.474925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.474985 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.475292 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.475293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.475294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.475439 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:15.475483 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:15.475904 (+ 421us) server_negotiation.cc:299] Negotiation successful
0926 13:06:15.475991 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:06:15.477026 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.463775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.3:48615)
0926 13:06:15.464314 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:15.464360 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:15.465059 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:15.465431 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:15.465436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:15.465705 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:15.466007 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.466014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.467175 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.467176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.468557 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.468561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.469183 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.469185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.469333 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.469998 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:15.470009 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:15.471767 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:15.474483 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.474491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.474494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.474828 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.475062 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.475062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.475063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.475208 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.475531 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:15.475538 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:15.475738 (+ 200us) client_negotiation.cc:769] Sending connection context
0926 13:06:15.476470 (+ 732us) client_negotiation.cc:240] Negotiation successful
0926 13:06:15.476722 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":95,"threads_started":1}
I20260926 13:06:15.477730 15108 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:15.477831 15108 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:15.478031 15108 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:15.478480 14615 ts_manager.cc:194] Registered new tserver with Master: a3d9d57ad0924ae284734bd25aef8ee7 (127.2.86.3:33587)
I20260926 13:06:15.478996 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:48615
I20260926 13:06:15.485069 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:06:15.485883 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 14705
I20260926 13:06:15.492205 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 14844
I20260926 13:06:15.497761 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 14979
I20260926 13:06:15.504854 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 14592
I20260926 13:06:15.512033 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40823
--webserver_interface=127.2.86.62
--webserver_port=39433
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40823
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:15.594574 15111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:15.594792 15111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:15.594815 15111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:15.595942 15111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:15.595984 15111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:15.595999 15111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:15.596011 15111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:15.596024 15111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:15.597702 15111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40823
--ranger_config_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=39433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:15.597944 15111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:15.598187 15111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:15.603262 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.603267 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:15.603538 15111 server_base.cc:1034] running on GCE node
W20260926 13:06:15.603267 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:15.603931 15111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:15.604290 15111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:15.605484 15111 hybrid_clock.cc:648] HybridClock initialized: now 1790427975605465 us; error 37 us; skew 500 ppm
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:15.608067 15111 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:15.608603 15111 webserver.cc:460] Webserver started at http://127.2.86.62:39433/ using document root <none> and password file <none>
I20260926 13:06:15.608839 15111 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:15.608937 15111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:15.651851 15111 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:15.652639 15127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.652827 15111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:15.652943 15111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "d4df3911fe5045f2ac616303909c1521"
format_stamp: "Formatted at 2026-09-26 13:06:13 on dist-test-slave-j4lq"
server_key: "b27701784aeb77600b1df80160c46fbe"
server_key_iv: "5d49163ee1f28898e679467019d89948"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.653369 15111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:15.674202 15111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:15.676437 15111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:15.676599 15111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:15.681213 15111 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40823
I20260926 13:06:15.681255 15179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40823 every 8 connection(s)
I20260926 13:06:15.681643 15111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:15.682937 15180 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:06:15.683676 15180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Bootstrap starting.
I20260926 13:06:15.687704 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15111
I20260926 13:06:15.688119 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:42699
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:40823
--webserver_port=45857
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:15.692543 15180 log.cc:824] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:15.694450 15180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:06:15.694768 15180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Bootstrap complete.
I20260926 13:06:15.697268 15180 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:15.697525 15180 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4df3911fe5045f2ac616303909c1521, State: Initialized, Role: FOLLOWER
I20260926 13:06:15.697721 15180 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:15.697817 15180 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:15.697871 15180 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:15.697919 15180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:06:15.698911 15180 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:15.699033 15180 leader_election.cc:302] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4df3911fe5045f2ac616303909c1521; no voters:
I20260926 13:06:15.699239 15180 leader_election.cc:288] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:06:15.699272 15184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:06:15.699385 15184 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [term 2 LEADER]: Becoming Leader. State: Replica: d4df3911fe5045f2ac616303909c1521, State: Running, Role: LEADER
I20260926 13:06:15.699509 15180 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:15.699545 15184 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } }
I20260926 13:06:15.699929 15185 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d4df3911fe5045f2ac616303909c1521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } } }
I20260926 13:06:15.700017 15185 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:15.700232 15186 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4df3911fe5045f2ac616303909c1521. Latest consensus state: current_term: 2 leader_uuid: "d4df3911fe5045f2ac616303909c1521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4df3911fe5045f2ac616303909c1521" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40823 } } }
I20260926 13:06:15.700312 15186 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:15.701002 15190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:15.701149 15180 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 13:06:15.701573 15190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:15.703682 15190 catalog_manager.cc:1259] Loaded cluster ID: cc4d4acc41274977bf5019aa34753e04
I20260926 13:06:15.703824 15190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:06:15.705912 15190 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:15.706183 15190 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Loaded TSK: 0
I20260926 13:06:15.706516 15190 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:06:15.802713 15182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:15.803022 15182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:15.803071 15182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:15.805106 15182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:15.805217 15182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:15.805284 15182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:15.807880 15182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:42699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=45857
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:15.808161 15182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:15.808508 15182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:15.815790 15218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.816133 15217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:15.816363 15220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:15.818536 15182 server_base.cc:1034] running on GCE node
I20260926 13:06:15.818912 15182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:15.819242 15182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:15.820464 15182 hybrid_clock.cc:648] HybridClock initialized: now 1790427975820428 us; error 53 us; skew 500 ppm
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:15.824106 15182 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:15.824712 15182 webserver.cc:460] Webserver started at http://127.2.86.1:45857/ using document root <none> and password file <none>
I20260926 13:06:15.825134 15182 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:15.825233 15182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:15.846657 15182 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 13:06:15.848937 15228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.849614 15182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260926 13:06:15.849713 15182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "21ae7ba2550248aa92eb2adecd8e6127"
format_stamp: "Formatted at 2026-09-26 13:06:14 on dist-test-slave-j4lq"
server_key: "1565de6b4747a66db817c7f2a4683b95"
server_key_iv: "7ca7f9d822514c4bd143d37c8c4a3d21"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:15.850157 15182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:15.861553 15182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:15.863916 15182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:15.864097 15182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:15.864373 15182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:15.864760 15182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:15.864814 15182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.864898 15182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:15.864934 15182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:15.872210 15182 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:42699
I20260926 13:06:15.872776 15182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:06:15.874047 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15182
I20260926 13:06:15.874439 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:36751
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:40823
--webserver_port=46813
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:15.876269 15341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:42699 every 8 connection(s)
Sep 26 13:06:15 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:15.904371 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.876999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.1:38931)
0926 13:06:15.877243 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:15.877275 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:15.877805 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:15.883751 (+ 5946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:15.883756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:15.884007 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:15.884300 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.884306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.886290 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.886291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.887633 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:15.887637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.888491 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.888492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:15.888622 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.889302 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:15.889312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:15.891026 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:15.901323 (+ 10297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.901336 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.901339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.901726 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.902870 (+ 1144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:15.902871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:15.902871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:15.903015 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:15.903580 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:15.903587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:15.903759 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 13:06:15.903865 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:06:15.904065 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260926 13:06:15.905655 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:15.878174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:38931 (local address 127.2.86.62:40823)
0926 13:06:15.878417 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:15.878426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:15.878460 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:06:15.878622 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:15.878630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:15.883457 (+ 4827us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:15.884448 (+ 991us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:15.884537 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.886145 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.887758 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:15.888364 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:15.888502 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:15.891174 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:15.891179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:15.891179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:15.891193 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:15.901178 (+ 9985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.902646 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.902650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.902650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.902780 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:15.903112 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:15.903114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:15.903114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:15.903486 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:15.905183 (+ 1697us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:15.905269 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:06:15.905412 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":115,"threads_started":1}
I20260926 13:06:15.907981 15342 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:15.908080 15342 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:15.908273 15342 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:15.909209 15342 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:40823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:06:15.909274 15342 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:15.909332 15342 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:15.909735 15342 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:15.909807 15342 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:15.910123 15342 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:06:16.008397 15344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:16.008701 15344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:16.008747 15344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:16.010813 15344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:16.010877 15344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:16.010919 15344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:16.013204 15344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:36751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=46813
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:16.013463 15344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:16.013751 15344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:16.019976 15352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:16.020638 15351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:16.021247 15354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:16.021402 15344 server_base.cc:1034] running on GCE node
I20260926 13:06:16.021622 15344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:16.021914 15344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:16.025183 15344 hybrid_clock.cc:648] HybridClock initialized: now 1790427976025160 us; error 36 us; skew 500 ppm
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:16.028673 15344 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:16.029152 15344 webserver.cc:460] Webserver started at http://127.2.86.2:46813/ using document root <none> and password file <none>
I20260926 13:06:16.029400 15344 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:16.029484 15344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:16.045454 15344 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 13:06:16.046418 15362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.046831 15344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:16.046968 15344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "a639a56e2913427b9bb90814a43dec68"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "f6d0b101ee9c0e53c6e44ec9c2f54fa5"
server_key_iv: "3f63bf66e30a4310aeddcb3b7611110f"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:16.047510 15344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:16.074728 15344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:16.077209 15344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:16.077373 15344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:16.077634 15344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:16.078102 15344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:16.078161 15344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.078192 15344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:16.078240 15344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.085561 15344 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:36751
I20260926 13:06:16.085647 15475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:36751 every 8 connection(s)
I20260926 13:06:16.086026 15344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:06:16.087749 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15344
I20260926 13:06:16.088101 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.86.20:35169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:33587
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:40823
--webserver_port=41805
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:16.096385 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:16.086817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:52289 (local address 127.2.86.62:40823)
0926 13:06:16.086895 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:16.086898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:16.087149 (+ 251us) server_negotiation.cc:407] Connection header received
0926 13:06:16.087932 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:16.087934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:16.087947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:16.088020 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:16.088777 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.089557 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.090700 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.090970 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.091006 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:16.093186 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:16.093187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:16.093187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:16.093195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:16.094791 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.095281 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.095282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.095282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.095341 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.095639 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.095640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.095640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.095762 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:16.095835 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:16.096199 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 13:06:16.096286 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:06:16.097131 15478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:16.086797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.2:52289)
0926 13:06:16.086976 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:16.087029 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:16.087726 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:16.088127 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:16.088132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:16.088378 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:16.088672 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:16.088679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.089666 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.089667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:16.090595 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:16.090599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.091099 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.091100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:16.091197 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:16.091752 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:16.091760 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:16.093080 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:16.094884 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:16.094889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:16.094892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:16.095189 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:16.095411 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:16.095411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:16.095412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:16.095541 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:16.095827 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:16.095834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:16.096021 (+ 187us) client_negotiation.cc:769] Sending connection context
0926 13:06:16.096720 (+ 699us) client_negotiation.cc:240] Negotiation successful
0926 13:06:16.096949 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260926 13:06:16.097594 15476 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:16.097687 15476 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.097867 15476 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:16.098501 15476 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:40823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:06:16.098574 15476 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.098637 15476 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:16.098973 15476 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.099043 15476 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:16.099377 15476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:06:16.210626 15479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:16.210968 15479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:16.211041 15479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:16.212980 15479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:16.213089 15479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:16.213166 15479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:16.215444 15479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.86.20:35169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:33587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=41805
--tserver_master_addrs=127.2.86.62:40823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:16.215870 15479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:16.216187 15479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:16.225769 15485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:16.225791 15486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:16.225786 15488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:16.226083 15479 server_base.cc:1034] running on GCE node
I20260926 13:06:16.226323 15479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:16.226646 15479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:16.227838 15479 hybrid_clock.cc:648] HybridClock initialized: now 1790427976227821 us; error 27 us; skew 500 ppm
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:16.231132 15479 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:16.231693 15479 webserver.cc:460] Webserver started at http://127.2.86.3:41805/ using document root <none> and password file <none>
I20260926 13:06:16.231992 15479 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:16.232100 15479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:16.247411 15479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:16.248325 15496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.248596 15479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:16.248714 15479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a3d9d57ad0924ae284734bd25aef8ee7"
format_stamp: "Formatted at 2026-09-26 13:06:15 on dist-test-slave-j4lq"
server_key: "3f263fc21bf376d757c7b96a95aa85fc"
server_key_iv: "61689925279a36cafe52bf5c603e7bf9"
server_key_version: "kuduclusterkey@0"
I20260926 13:06:16.249266 15479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:16.273438 15479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:16.275547 15479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:16.275720 15479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:16.275983 15479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:16.276368 15479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:16.276434 15479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.276480 15479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:16.276513 15479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:16.288661 15479 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:33587
I20260926 13:06:16.288702 15609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:33587 every 8 connection(s)
I20260926 13:06:16.289183 15479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:06:16.294788 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15479
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:16.299254 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:16.290071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40823 (local address 127.2.86.3:59303)
0926 13:06:16.290224 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:16.290260 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:16.290766 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:16.291022 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:16.291026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:16.291203 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:16.291397 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:16.291401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.292095 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.292096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:16.292965 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:16.292968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.293386 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.293387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:16.293502 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:16.294046 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:16.294054 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:16.295526 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:16.297580 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:16.297588 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:16.297590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:16.297898 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:16.298127 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:16.298128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:16.298128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:16.298265 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:16.298534 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:16.298540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:16.298705 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 13:06:16.298829 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 13:06:16.299008 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260926 13:06:16.300005 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:16.290070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:59303 (local address 127.2.86.62:40823)
0926 13:06:16.290123 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:16.290125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:16.290347 (+ 222us) server_negotiation.cc:407] Connection header received
0926 13:06:16.290883 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:16.290885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:16.290898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:16.290945 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:16.291499 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.291997 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.293054 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.293264 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.293351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:16.295659 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:16.295660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:16.295661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:16.295670 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:16.297490 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.297986 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.297987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.297988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.298050 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.298337 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.298338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.298338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.298464 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:16.298531 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:16.299825 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0926 13:06:16.299913 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:06:16.300575 15610 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40823
I20260926 13:06:16.300683 15610 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.300910 15610 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:16.301934 15610 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:40823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 13:06:16.302016 15610 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.302098 15610 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:16.302452 15610 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.302536 15610 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
W20260926 13:06:16.303211 15610 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:06:16.303645 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:16.295221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.2.86.62:40823)
0926 13:06:16.295882 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:16.295885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:16.295892 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:06:16.295911 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:16.295912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:16.295922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:16.296010 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:16.296782 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.297594 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.298554 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:16.298849 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:16.298949 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:16.300525 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:16.300526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:16.300526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:16.300532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:16.302400 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.302803 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.302805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.302805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.302861 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:16.303067 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:16.303068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:16.303069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:16.303208 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:16.303365 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:16.303451 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:06:16.303535 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":77,"threads_started":1}
Sep 26 13:06:16 dist-test-slave-j4lq krb5kdc[14148](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:16.910307 15342 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:16.910480 15342 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:16.910933 15144 ts_manager.cc:194] Registered new tserver with Master: 21ae7ba2550248aa92eb2adecd8e6127 (127.2.86.1:42699)
I20260926 13:06:16.911971 15144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:38931
I20260926 13:06:17.099691 15476 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:17.099833 15476 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:17.100248 15144 ts_manager.cc:194] Registered new tserver with Master: a639a56e2913427b9bb90814a43dec68 (127.2.86.2:36751)
I20260926 13:06:17.100834 15144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:52289
Sep 26 13:06:17 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
I20260926 13:06:17.303403 15610 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:17.303566 15610 heartbeater.cc:511] Master 127.2.86.62:40823 requested a full tablet report, sending...
I20260926 13:06:17.303941 15144 ts_manager.cc:194] Registered new tserver with Master: a3d9d57ad0924ae284734bd25aef8ee7 (127.2.86.3:33587)
I20260926 13:06:17.304481 15144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:59303
I20260926 13:06:17.307454 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:06:17.321424 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.308365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.2.86.62:40823)
0926 13:06:17.313273 (+ 4908us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.313276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.313288 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:06:17.313315 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.313316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.313329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.313421 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:17.314164 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.314921 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.316010 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.316292 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.316355 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.317963 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:17.317964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:17.317965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:17.317972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:17.319942 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:17.320414 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:17.320416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:17.320416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:17.320497 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:17.320742 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:17.320743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:17.320744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:17.320927 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:17.321036 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.321212 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.321311 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4881,"thread_start_us":63,"threads_started":1}
I20260926 13:06:17.323871 15144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:06:17.326545 15144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:06:17.335582 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.329770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42699 (local address 127.0.0.1:57744)
0926 13:06:17.330275 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.330318 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:17.330690 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:17.331019 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:17.331023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:17.331025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:17.331092 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.331094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.332697 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.332697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.333956 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.333958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.335237 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.335238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.335344 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.335359 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 13:06:17.335442 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:06:17.335497 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":71,"threads_started":1}
I20260926 13:06:17.335616 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.330215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.2.86.1:42699)
0926 13:06:17.330344 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.330348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.330390 (+ 42us) server_negotiation.cc:407] Connection header received
0926 13:06:17.330799 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.330801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.330817 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.330929 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:17.331176 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.332610 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.334060 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.335147 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.335230 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.335297 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.335481 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.335529 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":50,"threads_started":1}
I20260926 13:06:17.336951 15276 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc808f03ee54609b1940a47d073e689 (DEFAULT_TABLE table=test-table [id=bc9fec21528b44d9a6f31ac8717e9840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:17.337380 15276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc808f03ee54609b1940a47d073e689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:17.340225 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.329775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33587 (local address 127.0.0.1:40070)
0926 13:06:17.336046 (+ 6271us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.336058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:17.336110 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:17.336412 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:17.336412 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:17.336414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:17.336447 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.336448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.337948 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.337949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.339022 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.339024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.340002 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.340003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.340057 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.340060 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:06:17.340108 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 13:06:17.340147 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6257,"thread_start_us":97,"threads_started":1}
I20260926 13:06:17.340454 15641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.330051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.2.86.3:33587)
0926 13:06:17.330209 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.330214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.336177 (+ 5963us) server_negotiation.cc:407] Connection header received
0926 13:06:17.336223 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.336226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.336242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.336317 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:17.336512 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.337858 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.339111 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.339844 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.339904 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.339972 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.340331 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.340374 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260926 13:06:17.341588 15527 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc808f03ee54609b1940a47d073e689 (DEFAULT_TABLE table=test-table [id=bc9fec21528b44d9a6f31ac8717e9840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:17.341979 15527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc808f03ee54609b1940a47d073e689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:17.344696 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.330128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.2.86.2:36751)
0926 13:06:17.336978 (+ 6850us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.336984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.337166 (+ 182us) server_negotiation.cc:407] Connection header received
0926 13:06:17.337204 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.337206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.337221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.337288 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:17.337522 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.338590 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.339823 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.340570 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.340634 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.340689 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.344558 (+ 3869us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.344609 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6809,"thread_start_us":47,"threads_started":1}
I20260926 13:06:17.345117 15638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.330017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:36751 (local address 127.0.0.1:38950)
0926 13:06:17.337041 (+ 7024us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.337052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:17.337110 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:17.337400 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:17.337401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:17.337402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:17.337448 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.337450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.338684 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.338685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.339724 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.339727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.344374 (+ 4647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.344375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.344449 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.344453 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:17.344982 (+ 529us) client_negotiation.cc:240] Negotiation successful
0926 13:06:17.345035 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7009,"thread_start_us":74,"threads_started":1}
I20260926 13:06:17.346066 15394 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc808f03ee54609b1940a47d073e689 (DEFAULT_TABLE table=test-table [id=bc9fec21528b44d9a6f31ac8717e9840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:17.346444 15394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc808f03ee54609b1940a47d073e689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:17.347095 15645 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Bootstrap starting.
I20260926 13:06:17.347858 15645 tablet_bootstrap.cc:654] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:17.348270 15645 log.cc:824] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:17.349876 15647 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Bootstrap starting.
I20260926 13:06:17.350508 15644 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Bootstrap starting.
I20260926 13:06:17.350687 15645 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: No bootstrap required, opened a new log
I20260926 13:06:17.350795 15645 ts_tablet_manager.cc:1397] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260926 13:06:17.351315 15644 tablet_bootstrap.cc:654] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:17.351763 15644 log.cc:824] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:17.352656 15644 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: No bootstrap required, opened a new log
I20260926 13:06:17.352730 15644 ts_tablet_manager.cc:1397] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:17.354552 15645 raft_consensus.cc:348] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.354784 15645 raft_consensus.cc:374] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:17.354821 15645 raft_consensus.cc:729] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ae7ba2550248aa92eb2adecd8e6127, State: Initialized, Role: FOLLOWER
I20260926 13:06:17.354966 15645 consensus_queue.cc:260] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.355293 15645 ts_tablet_manager.cc:1428] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260926 13:06:17.355417 15342 heartbeater.cc:503] Master 127.2.86.62:40823 was elected leader, sending a full tablet report...
I20260926 13:06:17.356442 15644 raft_consensus.cc:348] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.356694 15644 raft_consensus.cc:374] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:17.356783 15644 raft_consensus.cc:729] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9d57ad0924ae284734bd25aef8ee7, State: Initialized, Role: FOLLOWER
I20260926 13:06:17.357005 15644 consensus_queue.cc:260] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.357080 15647 tablet_bootstrap.cc:654] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:17.357407 15644 ts_tablet_manager.cc:1428] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 13:06:17.357427 15647 log.cc:824] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:17.357595 15610 heartbeater.cc:503] Master 127.2.86.62:40823 was elected leader, sending a full tablet report...
I20260926 13:06:17.358922 15647 tablet_bootstrap.cc:492] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: No bootstrap required, opened a new log
I20260926 13:06:17.359025 15647 ts_tablet_manager.cc:1397] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260926 13:06:17.361483 15647 raft_consensus.cc:348] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.361673 15647 raft_consensus.cc:374] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:17.361783 15647 raft_consensus.cc:729] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a639a56e2913427b9bb90814a43dec68, State: Initialized, Role: FOLLOWER
I20260926 13:06:17.361948 15647 consensus_queue.cc:260] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.362247 15647 ts_tablet_manager.cc:1428] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:06:17.362258 15476 heartbeater.cc:503] Master 127.2.86.62:40823 was elected leader, sending a full tablet report...
W20260926 13:06:17.378077 15343 tablet.cc:2369] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 13:06:17 dist-test-slave-j4lq krb5kdc[14148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427976, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for HTTP/127.2.86.20@KRBTEST.COM
W20260926 13:06:17.540676 15611 tablet.cc:2369] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:06:17.587405 15477 tablet.cc:2369] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:06:17.618676 15649 raft_consensus.cc:482] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:17.618831 15649 raft_consensus.cc:504] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.619246 15649 leader_election.cc:288] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d9d57ad0924ae284734bd25aef8ee7 (127.2.86.3:33587), a639a56e2913427b9bb90814a43dec68 (127.2.86.2:36751)
I20260926 13:06:17.629755 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.619608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:34461 (local address 127.2.86.2:36751)
0926 13:06:17.619667 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.619670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.621086 (+ 1416us) server_negotiation.cc:407] Connection header received
0926 13:06:17.621238 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.621239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.621252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.621283 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:17.621498 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.622253 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.628608 (+ 6355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.629297 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.629357 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.629369 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.629632 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.629675 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:06:17.630280 15655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.619398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:36751 (local address 127.2.86.1:34461)
0926 13:06:17.620999 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.621012 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:17.621159 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:17.621348 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:17.621348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:17.621350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:17.621430 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.621431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.622362 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.622362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.623627 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.623631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.629462 (+ 5831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.629464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.629550 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.629554 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:17.630121 (+ 567us) client_negotiation.cc:240] Negotiation successful
0926 13:06:17.630180 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1578,"thread_start_us":93,"threads_started":1}
I20260926 13:06:17.630789 15430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc808f03ee54609b1940a47d073e689" candidate_uuid: "21ae7ba2550248aa92eb2adecd8e6127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a639a56e2913427b9bb90814a43dec68" is_pre_election: true
I20260926 13:06:17.631002 15430 raft_consensus.cc:2454] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21ae7ba2550248aa92eb2adecd8e6127 in term 0.
I20260926 13:06:17.631275 15232 leader_election.cc:302] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ae7ba2550248aa92eb2adecd8e6127, a639a56e2913427b9bb90814a43dec68; no voters:
I20260926 13:06:17.631431 15649 raft_consensus.cc:2781] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:06:17.631471 15649 raft_consensus.cc:482] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:06:17.631506 15649 raft_consensus.cc:3037] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:17.632403 15649 raft_consensus.cc:504] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.632536 15649 leader_election.cc:288] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [CANDIDATE]: Term 1 election: Requested vote from peers a3d9d57ad0924ae284734bd25aef8ee7 (127.2.86.3:33587), a639a56e2913427b9bb90814a43dec68 (127.2.86.2:36751)
I20260926 13:06:17.632921 15430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc808f03ee54609b1940a47d073e689" candidate_uuid: "21ae7ba2550248aa92eb2adecd8e6127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a639a56e2913427b9bb90814a43dec68"
I20260926 13:06:17.632999 15430 raft_consensus.cc:3037] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:17.633908 15430 raft_consensus.cc:2454] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21ae7ba2550248aa92eb2adecd8e6127 in term 1.
I20260926 13:06:17.634166 15232 leader_election.cc:302] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ae7ba2550248aa92eb2adecd8e6127, a639a56e2913427b9bb90814a43dec68; no voters:
I20260926 13:06:17.634284 15649 raft_consensus.cc:2781] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:17.634438 15649 raft_consensus.cc:686] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [term 1 LEADER]: Becoming Leader. State: Replica: 21ae7ba2550248aa92eb2adecd8e6127, State: Running, Role: LEADER
I20260926 13:06:17.634528 15649 consensus_queue.cc:237] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } }
I20260926 13:06:17.635421 15143 catalog_manager.cc:5495] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21ae7ba2550248aa92eb2adecd8e6127 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "21ae7ba2550248aa92eb2adecd8e6127" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21ae7ba2550248aa92eb2adecd8e6127" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 42699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 } health_report { overall_health: UNKNOWN } } }
I20260926 13:06:17.639933 15641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.619661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:55251 (local address 127.2.86.3:33587)
0926 13:06:17.619718 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.619721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.630717 (+ 10996us) server_negotiation.cc:407] Connection header received
0926 13:06:17.630818 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.630819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.630832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.630861 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:17.636482 (+ 5621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.637257 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.638652 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.639257 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.639343 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.639356 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.639806 (+ 450us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.639853 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:06:17.640671 15655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.619398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33587 (local address 127.2.86.1:55251)
0926 13:06:17.630663 (+ 11265us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.630673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:17.630747 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:17.630919 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:17.635949 (+ 5030us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:17.635953 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:17.636005 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.636008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.637370 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.637371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.638521 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:17.638524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.639630 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.639631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:17.639703 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.639707 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:17.640536 (+ 829us) client_negotiation.cc:240] Negotiation successful
0926 13:06:17.640591 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11246,"mutex_wait_us":13,"thread_start_us":81,"threads_started":1}
I20260926 13:06:17.641712 15563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc808f03ee54609b1940a47d073e689" candidate_uuid: "21ae7ba2550248aa92eb2adecd8e6127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" is_pre_election: true
I20260926 13:06:17.641887 15563 raft_consensus.cc:2454] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21ae7ba2550248aa92eb2adecd8e6127 in term 0.
I20260926 13:06:17.642027 15562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc808f03ee54609b1940a47d073e689" candidate_uuid: "21ae7ba2550248aa92eb2adecd8e6127" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9d57ad0924ae284734bd25aef8ee7"
I20260926 13:06:17.642124 15562 raft_consensus.cc:3037] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:17.642978 15562 raft_consensus.cc:2454] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21ae7ba2550248aa92eb2adecd8e6127 in term 1.
I20260926 13:06:17.690126 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.686777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.2.86.1:42699)
0926 13:06:17.686853 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:17.686856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:17.686864 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:06:17.686909 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:17.686910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:17.686935 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:17.687010 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:06:17.687208 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.687941 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.688999 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:17.689290 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:17.689352 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:17.689548 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:06:17.689689 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:06:17.689762 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:17.690005 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 13:06:17.690029 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 13:06:17.692433 15430 raft_consensus.cc:1261] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 1 FOLLOWER]: Refusing update from remote peer 21ae7ba2550248aa92eb2adecd8e6127: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:17.692749 15562 raft_consensus.cc:1261] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 1 FOLLOWER]: Refusing update from remote peer 21ae7ba2550248aa92eb2adecd8e6127: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:17.693328 15657 consensus_queue.cc:1035] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d9d57ad0924ae284734bd25aef8ee7" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:17.693500 15657 consensus_queue.cc:1035] T 6dc808f03ee54609b1940a47d073e689 P 21ae7ba2550248aa92eb2adecd8e6127 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a639a56e2913427b9bb90814a43dec68" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:17.699450 15660 mvcc.cc:205] Tried to move back new op lower bound from 7333592996625035264 to 7333592996391596032. Current Snapshot: MvccSnapshot[applied={T|T < 7333592996625035264}]
I20260926 13:06:17.700146 15663 mvcc.cc:205] Tried to move back new op lower bound from 7333592996625035264 to 7333592996391596032. Current Snapshot: MvccSnapshot[applied={T|T < 7333592996625035264}]
I20260926 13:06:17.702664 15143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:17.702766 15143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:17.703663 15143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4df3911fe5045f2ac616303909c1521: Sending DeleteTablet for 3 replicas of tablet 6dc808f03ee54609b1940a47d073e689
I20260926 13:06:17.704250 15527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc808f03ee54609b1940a47d073e689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:17 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:40070
I20260926 13:06:17.704512 15276 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc808f03ee54609b1940a47d073e689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:17 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:57744
I20260926 13:06:17.704726 15394 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc808f03ee54609b1940a47d073e689 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:17 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:38950
I20260926 13:06:17.705487 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15182
I20260926 13:06:17.709635 15668 tablet_replica.cc:330] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: stopping tablet replica
I20260926 13:06:17.709821 15670 tablet_replica.cc:330] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: stopping tablet replica
I20260926 13:06:17.710078 15670 raft_consensus.cc:2229] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:17.710244 15668 raft_consensus.cc:2229] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:17.710376 15668 pending_rounds.cc:62] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Trying to abort 2 pending ops.
I20260926 13:06:17.710491 15670 raft_consensus.cc:2258] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:17.710579 15668 pending_rounds.cc:68] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333592996625035264 op_type: WRITE_OP write_request { tablet_id: "6dc808f03ee54609b1940a47d073e689" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\365\210\337\325\006\"8\n\ntest-admin\022*\n bc9fec21528b44d9a6f31ac8717e9840\020\001\030\001 \001(\001" signature: "<redacted>""n\246\225\306\264Z\274)x\272\277\373&\253\253\355O\2272\347\021\371\273b.Pw\311\361{\242\035\030M\200\310\366\303mW\005\na\312\305\321Y=\t\264\315\027*&\003\342\276#\220-_\250\3477" signing_key_seq_num: 0 } } request_id { client_id: "1ad49eb3fb5e424f8c1a35bd4bec9649" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260926 13:06:17.711210 15668 pending_rounds.cc:68] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333592996391596032 op_type: NO_OP noop_request { }
I20260926 13:06:17.711338 15668 raft_consensus.cc:2866] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 13:06:17.711423 15668 raft_consensus.cc:2258] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:17.712980 15668 ts_tablet_manager.cc:1905] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:17.714902 15668 ts_tablet_manager.cc:1918] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:17.715042 15668 log.cc:1196] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/6dc808f03ee54609b1940a47d073e689
I20260926 13:06:17.715406 15668 ts_tablet_manager.cc:1939] T 6dc808f03ee54609b1940a47d073e689 P a3d9d57ad0924ae284734bd25aef8ee7: Deleting consensus metadata
I20260926 13:06:17.716116 15130 catalog_manager.cc:4841] TS a3d9d57ad0924ae284734bd25aef8ee7 (127.2.86.3:33587): tablet 6dc808f03ee54609b1940a47d073e689 (table test-table [id=bc9fec21528b44d9a6f31ac8717e9840]) successfully deleted
I20260926 13:06:17.718461 15670 ts_tablet_manager.cc:1905] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:17.720780 15670 ts_tablet_manager.cc:1918] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:17.720985 15670 log.cc:1196] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/6dc808f03ee54609b1940a47d073e689
I20260926 13:06:17.721360 15670 ts_tablet_manager.cc:1939] T 6dc808f03ee54609b1940a47d073e689 P a639a56e2913427b9bb90814a43dec68: Deleting consensus metadata
I20260926 13:06:17.721915 15131 catalog_manager.cc:4841] TS a639a56e2913427b9bb90814a43dec68 (127.2.86.2:36751): tablet 6dc808f03ee54609b1940a47d073e689 (table test-table [id=bc9fec21528b44d9a6f31ac8717e9840]) successfully deleted
W20260926 13:06:17.723580 15499 connection.cc:535] server connection from 127.2.86.1:55251 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 13:06:17.723923 15131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:42699 (error 108)
I20260926 13:06:17.724970 15638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:17.724819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:42699 (local address 127.0.0.1:57762)
0926 13:06:17.724889 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:17.724934 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:42699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 13:06:17.725225 15131 catalog_manager.cc:4584] TS 21ae7ba2550248aa92eb2adecd8e6127 (127.2.86.1:42699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dc808f03ee54609b1940a47d073e689: Network error: Client connection negotiation failed: client connection to 127.2.86.1:42699: connect: Connection refused (error 111)
I20260926 13:06:17.725463 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15344
I20260926 13:06:17.732268 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15479
I20260926 13:06:17.738446 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15111
I20260926 13:06:17.744683 2392 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 13:06:18.090078 2392 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 13:06:18.090195 2392 mini_ranger.cc:66] Stopping Ranger...
I20260926 13:06:18.435632 2392 mini_ranger.cc:68] Stopped Ranger
2026-09-26 13:06:18.435 UTC [14178] LOG: received smart shutdown request
2026-09-26 13:06:18.439 UTC [14178] LOG: background worker "logical replication launcher" (PID 14187) exited with exit code 1
2026-09-26 13:06:18.440 UTC [14182] LOG: shutting down
2026-09-26 13:06:18.620 UTC [14178] LOG: database system is shut down
2026-09-26T13:06:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49141 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:06:18 dist-test-slave-j4lq krb5kdc[15683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:06:18 dist-test-slave-j4lq krb5kdc[15683](info): set up 2 sockets
Sep 26 13:06:18 dist-test-slave-j4lq krb5kdc[15683](info): commencing operation
krb5kdc: starting...
W20260926 13:06:19.776734 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:06:19 dist-test-slave-j4lq krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:06:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:06:19.896531 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:44465
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:19.977382 15696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:19.977659 15696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:19.977701 15696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:19.978885 15696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:19.978964 15696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:19.978989 15696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:19.979010 15696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:19.979040 15696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:19.980608 15696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:19.980932 15696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:19.981174 15696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:19.985996 15701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:19.985996 15702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:19.986238 15696 server_base.cc:1034] running on GCE node
W20260926 13:06:19.986173 15704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:19.986548 15696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:19.986874 15696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:19.988050 15696 hybrid_clock.cc:648] HybridClock initialized: now 1790427979988044 us; error 42 us; skew 500 ppm
Sep 26 13:06:19 dist-test-slave-j4lq krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:19.990706 15696 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:19.991187 15696 webserver.cc:460] Webserver started at http://127.2.86.62:41557/ using document root <none> and password file <none>
I20260926 13:06:19.991379 15696 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:19.991418 15696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:19.991511 15696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:19.992573 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "02c77569dbdf44089b0f588f9e78607c"
format_stamp: "Formatted at 2026-09-26 13:06:19 on dist-test-slave-j4lq"
server_key: "57c701869e58dc061c8ff12d66a269fc"
server_key_iv: "51a74b8273cc29c28cf04143073d88d9"
server_key_version: "encryptionkey@0"
I20260926 13:06:19.992947 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "02c77569dbdf44089b0f588f9e78607c"
format_stamp: "Formatted at 2026-09-26 13:06:19 on dist-test-slave-j4lq"
server_key: "57c701869e58dc061c8ff12d66a269fc"
server_key_iv: "51a74b8273cc29c28cf04143073d88d9"
server_key_version: "encryptionkey@0"
I20260926 13:06:19.994271 15696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:19.995052 15711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:19.995239 15696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:19.995337 15696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "02c77569dbdf44089b0f588f9e78607c"
format_stamp: "Formatted at 2026-09-26 13:06:19 on dist-test-slave-j4lq"
server_key: "57c701869e58dc061c8ff12d66a269fc"
server_key_iv: "51a74b8273cc29c28cf04143073d88d9"
server_key_version: "encryptionkey@0"
I20260926 13:06:19.995393 15696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:20.008355 15696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:20.010777 15696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:20.010896 15696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:20.014793 15696 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:44465
I20260926 13:06:20.014848 15763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:44465 every 8 connection(s)
I20260926 13:06:20.015211 15696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:20.016121 15764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:20.017791 15764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:06:20.021203 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15696
I20260926 13:06:20.021292 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:06:20.021495 2392 external_mini_cluster.cc:1428] Setting key 7ded2bacb472f62c36a5db074c8843d6
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:20.028695 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.018489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:44465 (local address 127.0.0.1:33364)
0926 13:06:20.018659 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.018701 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.019221 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.019522 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.019525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.019811 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.019963 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.019967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.021106 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.021107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.021967 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.021969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.022500 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.022501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.022565 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.023213 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.023225 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.024834 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.027131 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.027139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.027141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.027346 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.027640 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.027641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.027641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.027721 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.028179 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.028183 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.028242 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.028315 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.028449 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260926 13:06:20.028710 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.018514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.2.86.62:44465)
0926 13:06:20.018695 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.018706 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.018822 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:06:20.019387 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.019389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.019409 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.019519 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.020083 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.020986 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.022065 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.022400 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.022529 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.025000 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.025008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.025008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.025015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.027031 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.027439 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.027443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.027443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.027555 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.027790 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.027791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.027791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.028102 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.028199 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.028375 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.028532 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260926 13:06:20.029626 15764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.86.61@KRBTEST.COM; defaulting to no policy
I20260926 13:06:20.030002 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.029876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59676)
0926 13:06:20.029928 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.029972 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 13:06:20.030135 15712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
I20260926 13:06:20.030575 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.030460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59688)
0926 13:06:20.030515 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.030546 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:06:20.030669 15764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:38023: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
I20260926 13:06:20.047783 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:38023
--webserver_interface=127.2.86.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:20.079890 15764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } attempt: 1
I20260926 13:06:20.080403 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.080267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59700)
0926 13:06:20.080329 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.080373 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:06:20.080747 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.080651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59702)
0926 13:06:20.080698 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.080722 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 13:06:20.080833 15764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:38023: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
W20260926 13:06:20.129298 15771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:20.129551 15771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:20.129599 15771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:20.130765 15771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:20.130841 15771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:20.130863 15771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:20.130884 15771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:20.130913 15771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:20.132442 15771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:38023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.86.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:20.132735 15771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:20.132990 15771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:20.137686 15777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:20.137682 15779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:20.137749 15776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:20.137924 15771 server_base.cc:1034] running on GCE node
I20260926 13:06:20.138175 15771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:20.138487 15771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:20.139657 15771 hybrid_clock.cc:648] HybridClock initialized: now 1790427980139652 us; error 43 us; skew 500 ppm
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:20.142295 15771 init.cc:377] Logged in from keytab as kudu/127.2.86.61@KRBTEST.COM (short username kudu)
I20260926 13:06:20.142735 15771 webserver.cc:460] Webserver started at http://127.2.86.61:34377/ using document root <none> and password file <none>
I20260926 13:06:20.142972 15771 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:20.143039 15771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:20.143193 15771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:20.144255 15771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data/instance:
uuid: "8dd24bef4ce14e3ba07168ef9e0aa104"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "b56594118a7e726aa9562784f0e72866"
server_key_iv: "ab8c0ed5f3317207d73befb479928c92"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.144613 15771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal/instance:
uuid: "8dd24bef4ce14e3ba07168ef9e0aa104"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "b56594118a7e726aa9562784f0e72866"
server_key_iv: "ab8c0ed5f3317207d73befb479928c92"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.145968 15771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:06:20.146674 15786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:20.146837 15771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:06:20.146937 15771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "8dd24bef4ce14e3ba07168ef9e0aa104"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "b56594118a7e726aa9562784f0e72866"
server_key_iv: "ab8c0ed5f3317207d73befb479928c92"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.147020 15771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:20.148931 15764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } attempt: 2
I20260926 13:06:20.149309 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.149199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59710)
0926 13:06:20.149243 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.149278 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 13:06:20.149658 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.149552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59720)
0926 13:06:20.149598 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.149626 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 13:06:20.149765 15764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:38023: Network error: Client connection negotiation failed: client connection to 127.2.86.61:38023: connect: Connection refused (error 111)
I20260926 13:06:20.165026 15771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:20.167402 15771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:20.167567 15771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:20.171394 15771 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.61:38023
I20260926 13:06:20.171455 15838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.61:38023 every 8 connection(s)
I20260926 13:06:20.171797 15771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
I20260926 13:06:20.172606 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15771
I20260926 13:06:20.172693 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-1/wal/instance
I20260926 13:06:20.172755 15839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:20.172935 2392 external_mini_cluster.cc:1428] Setting key 9f4fbe3ba0545840837c0daedacd024c
I20260926 13:06:20.174440 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.86.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.60@KRBTEST.COM" created.
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:20.187564 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.175240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.2.86.62:44465)
0926 13:06:20.175311 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.175313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.175557 (+ 244us) server_negotiation.cc:407] Connection header received
0926 13:06:20.176318 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.176320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.176338 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.176411 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.177284 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.177770 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.179307 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.179501 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.179553 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.184462 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.184464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.184464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.184472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.186179 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.186560 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.186561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.186561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.186630 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.186858 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.186859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.186860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.187016 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.187100 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.187333 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.187432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:06:20.187603 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.175174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:44465 (local address 127.0.0.1:33370)
0926 13:06:20.175392 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.175439 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.176181 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.176491 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.176495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.176859 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.177190 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.177197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.177866 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.177867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.179219 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.179223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.179586 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.179587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.179735 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.180278 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.180289 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.184369 (+ 4080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.186266 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.186272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.186274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.186488 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.186698 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.186699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.186699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.186794 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.187082 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.187086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.187205 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.187284 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.187397 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260926 13:06:20.188202 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:06:20.197693 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.188474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59730)
0926 13:06:20.188522 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.188532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.188571 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.188943 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.188944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.189066 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.189117 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.189119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.190476 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.190477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.190975 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.190976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.191639 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.191640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.191692 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.192111 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.192116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.194864 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.196676 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.196677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.196677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.196830 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.197118 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.197119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.197120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.197185 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.197414 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.197415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.197449 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.197499 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.197611 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 13:06:20.197803 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.188567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.2.86.61:38023)
0926 13:06:20.188761 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.188767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.188792 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:06:20.188819 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.188821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.188836 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.188943 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.189193 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.190387 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.191058 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.191558 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.191646 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.194928 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.194936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.194936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.194942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.196582 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.196949 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.196952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.196952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.197004 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.197228 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.197229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.197229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.197361 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.197421 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.197549 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.197708 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260926 13:06:20.198432 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:06:20.198798 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.198692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43836)
0926 13:06:20.198732 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.198774 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260926 13:06:20.198956 15787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
I20260926 13:06:20.199486 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.199412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43852)
0926 13:06:20.199449 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.199471 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 13:06:20.199606 15839 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:33857: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
I20260926 13:06:20.200491 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:33857
--webserver_interface=127.2.86.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:20.240963 15764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } attempt: 3
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:06:20.248137 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.241467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.2.86.61:38023)
0926 13:06:20.241532 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.241534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.241547 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:06:20.241635 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.241636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.241646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.241715 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.242033 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.242528 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.243222 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.243421 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.243506 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.245701 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.245703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.245703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.245710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.247104 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.247471 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.247472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.247472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.247519 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.247703 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.247704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.247704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.247802 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.247855 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.248001 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.248061 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:06:20.248171 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.241391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59736)
0926 13:06:20.241455 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.241473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.241553 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.241720 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.241721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.241883 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.241949 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.241951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.242625 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.242627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.243139 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.243140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.243505 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.243506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.243584 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.244101 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.244106 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.245618 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.247187 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.247188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.247188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.247396 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.247580 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.247581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.247581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.247645 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.247860 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.247861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.247910 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.247979 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.248076 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 13:06:20.248627 15764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:06:20.248790 15839 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } attempt: 1
I20260926 13:06:20.248978 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.248846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43868)
0926 13:06:20.248913 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.248950 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 13:06:20.249156 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.249059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43880)
0926 13:06:20.249099 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.249130 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 13:06:20.249331 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.249253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43886)
0926 13:06:20.249284 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.249311 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260926 13:06:20.249404 15764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:33857: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
I20260926 13:06:20.249423 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.249355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43890)
0926 13:06:20.249379 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.249403 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260926 13:06:20.249511 15839 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:33857: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
I20260926 13:06:20.280546 15764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } attempt: 1
I20260926 13:06:20.281179 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.281034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43896)
0926 13:06:20.281104 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.281139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 13:06:20.281628 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.281486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43906)
0926 13:06:20.281537 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.281604 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 13:06:20.281735 15764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:33857: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
W20260926 13:06:20.283102 15846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:20.283293 15846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:20.283344 15846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:20.284513 15846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:20.284557 15846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:20.284572 15846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:20.284585 15846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:20.284597 15846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:20.286217 15846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:44465,127.2.86.61:38023,127.2.86.60:33857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:33857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.86.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:20.286470 15846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:20.286721 15846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:20.291640 15851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:20.291689 15854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:20.291620 15852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:20.291692 15846 server_base.cc:1034] running on GCE node
I20260926 13:06:20.292023 15846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:20.292292 15846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:20.293457 15846 hybrid_clock.cc:648] HybridClock initialized: now 1790427980293445 us; error 27 us; skew 500 ppm
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:20.295888 15846 init.cc:377] Logged in from keytab as kudu/127.2.86.60@KRBTEST.COM (short username kudu)
I20260926 13:06:20.296319 15846 webserver.cc:460] Webserver started at http://127.2.86.60:43289/ using document root <none> and password file <none>
I20260926 13:06:20.296555 15846 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:20.296630 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:20.296775 15846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:20.297914 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data/instance:
uuid: "2234c9d6b1b2478e881838e0752f220a"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "ff96900924b6e82eb46215444accfe13"
server_key_iv: "942a9ea3b612fd973b4a4320cef334a8"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.298283 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal/instance:
uuid: "2234c9d6b1b2478e881838e0752f220a"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "ff96900924b6e82eb46215444accfe13"
server_key_iv: "942a9ea3b612fd973b4a4320cef334a8"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.299566 15846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:06:20.300256 15861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:20.300419 15846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:06:20.300513 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "2234c9d6b1b2478e881838e0752f220a"
format_stamp: "Formatted at 2026-09-26 13:06:20 on dist-test-slave-j4lq"
server_key: "ff96900924b6e82eb46215444accfe13"
server_key_iv: "942a9ea3b612fd973b4a4320cef334a8"
server_key_version: "encryptionkey@0"
I20260926 13:06:20.300588 15846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:20.317704 15839 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } attempt: 2
I20260926 13:06:20.318186 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.318039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43916)
0926 13:06:20.318104 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.318150 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 13:06:20.318562 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.318478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43922)
0926 13:06:20.318519 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.318543 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 13:06:20.318694 15839 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:33857: Network error: Client connection negotiation failed: client connection to 127.2.86.60:33857: connect: Connection refused (error 111)
I20260926 13:06:20.338270 15846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:20.340728 15846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:20.340909 15846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:20.344694 15846 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.60:33857
I20260926 13:06:20.344748 15913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.60:33857 every 8 connection(s)
I20260926 13:06:20.345147 15846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
I20260926 13:06:20.345297 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 15846
I20260926 13:06:20.345404 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790427826505950-2392-0/minicluster-data/master-2/wal/instance
I20260926 13:06:20.345613 2392 external_mini_cluster.cc:1428] Setting key d5bcba230e9cc2049e483f6e60e6d439
I20260926 13:06:20.346191 15914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:20.348323 15914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:20.356597 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.346125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.2.86.60:33857)
0926 13:06:20.346311 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.346321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.346357 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:06:20.346781 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.346787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.346831 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.346943 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.347725 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.348652 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.349664 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.350212 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.350289 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.353086 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.353094 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.353094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.353107 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.355039 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.355418 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.355422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.355422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.355508 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.355721 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.355722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.355723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.355951 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.356115 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.356212 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.356340 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260926 13:06:20.356662 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.349005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.2.86.62:44465)
0926 13:06:20.349053 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.349055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.349247 (+ 192us) server_negotiation.cc:407] Connection header received
0926 13:06:20.349312 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.349313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.349324 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.349375 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.349846 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.350446 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.351483 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.351809 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.351880 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.354026 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.354028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.354028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.354036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.355448 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.355831 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.355832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.355833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.355891 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.356097 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.356097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.356098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.356206 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.356281 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.356472 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.356549 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:06:20.356604 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.348978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:44465 (local address 127.0.0.1:33386)
0926 13:06:20.349138 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.349176 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.349249 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.349398 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.349400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.349699 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.349777 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.349778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.350561 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.350562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.351393 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.351396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.351897 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.351899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.351982 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.352438 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.352447 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.353932 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.355543 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.355550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.355552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.355755 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.355960 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.355961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.355961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.356019 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.356274 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.356280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.356331 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.356424 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.356526 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260926 13:06:20.356859 15764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } attempt: 2
I20260926 13:06:20.357553 15914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:20.365482 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.357895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.2.86.61:38023)
0926 13:06:20.357935 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.357937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.357944 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:06:20.357973 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.357974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.357981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.358047 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.358337 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.359033 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359973 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.360263 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.360317 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.362243 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.362244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.362245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.362253 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.363880 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.364337 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.364338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.364338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.364395 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.364609 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.364610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.364611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.364752 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.364823 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.365315 (+ 492us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.365385 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:06:20.365839 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.357179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43944)
0926 13:06:20.357220 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.357232 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.357273 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.357488 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.357489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.357667 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.357712 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.357715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.358671 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.358672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.359224 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.359226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.359683 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.359759 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.360413 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.360418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.362648 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.364827 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.364828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.364828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.365048 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.365246 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.365247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.365247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.365303 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.365546 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.365547 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.365588 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.365650 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.365739 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 13:06:20.366120 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.357331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.2.86.60:33857)
0926 13:06:20.357364 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.357366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.357373 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:06:20.357390 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.357391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.357400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.357475 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.357770 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.358594 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359303 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.359596 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359663 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.363014 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.363015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.363016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.363022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.364747 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.365130 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.365131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.365131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.365171 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.365373 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.365373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.365374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.365479 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.365553 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.365769 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.365883 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":16}
I20260926 13:06:20.366138 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.357788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:38023 (local address 127.0.0.1:59742)
0926 13:06:20.357820 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.357830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.357861 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.358050 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.358051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.358192 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.358238 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.358239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.359128 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.359873 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.359874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.360339 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.360340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.360409 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.360755 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.360760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.362155 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.363974 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.363975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.363975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.364252 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.364472 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.364473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.364473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.364537 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.365143 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.365144 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.365189 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.365693 (+ 504us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.365824 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 13:06:20.366446 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.357566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.2.86.62:44465)
0926 13:06:20.357597 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.357598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.357624 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:06:20.357721 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.357722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.357733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.357783 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.358298 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.359013 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.359780 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.360067 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.360142 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.362890 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.362891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.362892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.362898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.364528 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.364863 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.364890 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.364890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.364942 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.365149 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.365150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.365151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.365285 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.366048 (+ 763us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.366280 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.366355 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:06:20.366725 15914 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:06:20.367763 15764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: Bootstrap starting.
I20260926 13:06:20.368417 15764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:20.368716 15764 log.cc:824] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:20.369532 15764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: No bootstrap required, opened a new log
I20260926 13:06:20.371542 15764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.371706 15764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:20.371763 15764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c77569dbdf44089b0f588f9e78607c, State: Initialized, Role: FOLLOWER
I20260926 13:06:20.371901 15764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.372234 15764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:20.372251 15920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.372355 15920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: This master's current role is: FOLLOWER
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
W20260926 13:06:20.374795 15931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:06:20.374847 15931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:06:20.376439 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.366967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43960)
0926 13:06:20.367014 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.367023 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.367058 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.369729 (+ 2671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.369730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.369840 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.369868 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.369869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.370475 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.370475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.370935 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.370936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.371275 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.371276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.371319 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.371900 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.371906 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.373341 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.375322 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.375323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.375324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.375535 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.375739 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.375740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.375740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.375796 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.376131 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.376132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.376170 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.376229 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.376321 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 13:06:20.376726 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.367015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.2.86.60:33857)
0926 13:06:20.369091 (+ 2076us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.369093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.369102 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:06:20.369124 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.369124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.369134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.369198 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.369946 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.370388 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.371013 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.371206 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.371280 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.373449 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.373451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.373451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.373458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.375238 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.375621 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.375622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.375623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.375676 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.375862 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.375862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.375863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.376046 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.376110 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.376540 (+ 430us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.376622 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2057}
I20260926 13:06:20.375084 15696 master_runner.cc:186] Error getting master registration for 127.2.86.61:38023: OK, instance_id { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" instance_seqno: 1790427980170302 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:06:20.377534 15696 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:06:20.377678 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.369028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59744 (local address 127.2.86.61:38023)
0926 13:06:20.369084 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.369086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.369103 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:06:20.369125 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.369125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.369134 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.369198 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.369713 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.370400 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.371360 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.371626 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.371669 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.374648 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.374650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.374650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.374658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.376466 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.376844 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.376846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.376846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.376961 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.377161 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.377162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.377162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.377269 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.377468 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.377503 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.377580 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:06:20.379117 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:06:20.379417 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: Bootstrap starting.
I20260926 13:06:20.380015 15914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:20.380354 15914 log.cc:824] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:20.381312 15914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: No bootstrap required, opened a new log
I20260926 13:06:20.383085 15914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.383224 15914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:20.383267 15914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2234c9d6b1b2478e881838e0752f220a, State: Initialized, Role: FOLLOWER
I20260926 13:06:20.383391 15914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.383642 15941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.383706 15914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:20.383783 15941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:06:20.385668 15952 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:06:20.385774 15952 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:06:20.388823 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.380982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.2.86.61:38023)
0926 13:06:20.381018 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.381020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.381264 (+ 244us) server_negotiation.cc:407] Connection header received
0926 13:06:20.381313 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.381314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.381326 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.381386 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.382058 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.382768 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.383812 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.384104 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.384147 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.385809 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.385811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.385811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.385818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.387550 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.387951 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.387952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.387953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.388003 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.388216 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.388217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.388218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.388337 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.388425 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.388649 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.388727 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:06:20.386902 15846 master_runner.cc:186] Error getting master registration for 127.2.86.61:38023: OK, instance_id { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" instance_seqno: 1790427980170302 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:06:20.389997 15846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:06:20.393332 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.380580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.2.86.62:44465)
0926 13:06:20.380627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.380629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.380901 (+ 272us) server_negotiation.cc:407] Connection header received
0926 13:06:20.385239 (+ 4338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.385241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.385251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.385282 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.385918 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.387057 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.388022 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.388308 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.388356 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.390240 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.390241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.390241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.390249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.392096 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.392436 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.392437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.392437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.392506 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.392721 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.392722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.392722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.392839 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.393035 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.393147 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.393219 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 13:06:20.394640 15941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:20.394762 15941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.395012 15941 leader_election.cc:288] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c77569dbdf44089b0f588f9e78607c (127.2.86.62:44465), 8dd24bef4ce14e3ba07168ef9e0aa104 (127.2.86.61:38023)
I20260926 13:06:20.395274 15738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2234c9d6b1b2478e881838e0752f220a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c77569dbdf44089b0f588f9e78607c" is_pre_election: true
I20260926 13:06:20.395426 15738 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2234c9d6b1b2478e881838e0752f220a in term 0.
I20260926 13:06:20.395644 15862 leader_election.cc:302] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c77569dbdf44089b0f588f9e78607c, 2234c9d6b1b2478e881838e0752f220a; no voters:
I20260926 13:06:20.395797 15941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:06:20.395861 15941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:06:20.395890 15941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:20.396770 15941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.396944 15941 leader_election.cc:288] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [CANDIDATE]: Term 1 election: Requested vote from peers 02c77569dbdf44089b0f588f9e78607c (127.2.86.62:44465), 8dd24bef4ce14e3ba07168ef9e0aa104 (127.2.86.61:38023)
I20260926 13:06:20.397195 15738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2234c9d6b1b2478e881838e0752f220a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c77569dbdf44089b0f588f9e78607c"
I20260926 13:06:20.397295 15738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:20.397845 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.381014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.2.86.60:33857)
0926 13:06:20.381047 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.381049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.389488 (+ 8439us) server_negotiation.cc:407] Connection header received
0926 13:06:20.389578 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.389579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.389588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.389628 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.390279 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.391764 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.392739 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.393081 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.393124 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.394793 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.394794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.394794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.394800 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.396516 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.396947 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.396948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.396948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.397027 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.397233 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.397234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.397235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.397386 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.397475 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.397662 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.397734 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 13:06:20.398151 15738 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2234c9d6b1b2478e881838e0752f220a in term 1.
I20260926 13:06:20.398388 15862 leader_election.cc:302] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c77569dbdf44089b0f588f9e78607c, 2234c9d6b1b2478e881838e0752f220a; no voters:
I20260926 13:06:20.398550 15941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:20.398700 15941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [term 1 LEADER]: Becoming Leader. State: Replica: 2234c9d6b1b2478e881838e0752f220a, State: Running, Role: LEADER
I20260926 13:06:20.398780 15941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.399222 15954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2234c9d6b1b2478e881838e0752f220a. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.399466 15954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:20.399716 15956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:20.399900 15956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:20.401412 15738 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [term 1 FOLLOWER]: Refusing update from remote peer 2234c9d6b1b2478e881838e0752f220a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:20.401784 15954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:20.403049 15920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2234c9d6b1b2478e881838e0752f220a. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.403134 15920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:06:20.403892 15954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.403981 15954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:20.404532 15920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.404623 15920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:06:20.404726 15957 mvcc.cc:205] Tried to move back new op lower bound from 7333593007722000384 to 7333593007714037760. Current Snapshot: MvccSnapshot[applied={T|T < 7333593007722000384}]
I20260926 13:06:20.404846 15958 mvcc.cc:205] Tried to move back new op lower bound from 7333593007722000384 to 7333593007714037760. Current Snapshot: MvccSnapshot[applied={T|T < 7333593007722000384}]
I20260926 13:06:20.405607 15956 catalog_manager.cc:1347] Generated new cluster ID: 848eb223d9724307b40ef68b16ad9af7
I20260926 13:06:20.405651 15956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:06:20.409847 15839 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } attempt: 3
Sep 26 13:06:20 dist-test-slave-j4lq krb5kdc[15683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
I20260926 13:06:20.415867 15956 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:06:20.416646 15956 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:20.417918 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.410089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:33857 (local address 127.0.0.1:43986)
0926 13:06:20.410159 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:20.410169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:20.410202 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:20.410452 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:20.410453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:20.410623 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:20.410666 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.410667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.411533 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.411534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.412312 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:20.412313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.412807 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.412808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:20.412912 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.413416 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:20.413421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:20.414903 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:20.416764 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.416765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.416766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.417041 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.417281 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:20.417282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:20.417282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:20.417348 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:20.417629 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:20.417630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:20.417683 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 13:06:20.417737 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:06:20.417823 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 13:06:20.418280 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:20.410277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.2.86.60:33857)
0926 13:06:20.410308 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:20.410310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:20.410316 (+ 6us) server_negotiation.cc:407] Connection header received
0926 13:06:20.410333 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:20.410334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:20.410342 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:20.410384 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:20.410734 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.411453 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.412404 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:20.412692 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:20.412743 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:20.415015 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:20.415017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:20.415017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:20.415024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:20.416644 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.417133 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.417134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.417134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.417204 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:20.417416 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:20.417417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:20.417417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:20.417537 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:20.417588 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:20.418112 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 13:06:20.418198 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:06:20.419658 15839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: Bootstrap starting.
I20260926 13:06:20.420315 15839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:20.420589 15839 log.cc:824] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:20.421451 15839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: No bootstrap required, opened a new log
I20260926 13:06:20.423249 15839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.423374 15839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:20.423403 15839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd24bef4ce14e3ba07168ef9e0aa104, State: Initialized, Role: FOLLOWER
I20260926 13:06:20.423553 15839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } }
I20260926 13:06:20.423945 15963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.424091 15839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:20.424142 15812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2234c9d6b1b2478e881838e0752f220a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104"
I20260926 13:06:20.424198 15811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:20.425657 15811 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 1 FOLLOWER]: Refusing update from remote peer 2234c9d6b1b2478e881838e0752f220a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:20.426075 15954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:20.427210 15956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a: Generated new TSK 0
I20260926 13:06:20.427383 15956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:06:20.424212 15813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2234c9d6b1b2478e881838e0752f220a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" is_pre_election: true
I20260926 13:06:20.427457 15813 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2234c9d6b1b2478e881838e0752f220a in term 1.
I20260926 13:06:20.429440 15941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.429553 15941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2234c9d6b1b2478e881838e0752f220a [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:20.429600 15978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.429914 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2234c9d6b1b2478e881838e0752f220a. Latest consensus state: current_term: 1 leader_uuid: "2234c9d6b1b2478e881838e0752f220a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c77569dbdf44089b0f588f9e78607c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 44465 } } peers { permanent_uuid: "8dd24bef4ce14e3ba07168ef9e0aa104" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 38023 } } peers { permanent_uuid: "2234c9d6b1b2478e881838e0752f220a" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 33857 } } }
I20260926 13:06:20.430343 15978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:06:20.430375 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:06:20.424116 15963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:06:20.430971 15974 catalog_manager.cc:1259] Loaded cluster ID: 848eb223d9724307b40ef68b16ad9af7
I20260926 13:06:20.431016 15974 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: loading cluster ID for follower catalog manager: success
I20260926 13:06:20.432158 15974 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: acquiring CA information for follower catalog manager: success
I20260926 13:06:20.432471 15974 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8dd24bef4ce14e3ba07168ef9e0aa104: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:06:21.375257 15931 catalog_manager.cc:1259] Loaded cluster ID: 848eb223d9724307b40ef68b16ad9af7
I20260926 13:06:21.375334 15931 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: loading cluster ID for follower catalog manager: success
I20260926 13:06:21.376194 15931 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: acquiring CA information for follower catalog manager: success
I20260926 13:06:21.376515 15931 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02c77569dbdf44089b0f588f9e78607c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:06:21.485069 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15696
I20260926 13:06:21.490389 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15771
I20260926 13:06:21.495187 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 15846
2026-09-26T13:06:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2763 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:06:21 dist-test-slave-j4lq krb5kdc[15989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:06:21 dist-test-slave-j4lq krb5kdc[15989](info): set up 2 sockets
Sep 26 13:06:21 dist-test-slave-j4lq krb5kdc[15989](info): commencing operation
krb5kdc: starting...
W20260926 13:06:22.541019 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[15989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:06:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:06:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:06:22.659917 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40619
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:22.742089 16002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:22.742323 16002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:22.742348 16002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:22.743592 16002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:22.743643 16002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:22.743659 16002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:22.743671 16002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:22.743683 16002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:22.745270 16002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:22.745554 16002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:22.745810 16002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:22.750774 16008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:22.750759 16007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:22.750787 16010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:22.751157 16002 server_base.cc:1034] running on GCE node
I20260926 13:06:22.751430 16002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:22.751744 16002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:22.752962 16002 hybrid_clock.cc:648] HybridClock initialized: now 1790427982752945 us; error 32 us; skew 500 ppm
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[15989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427982, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:22.755679 16002 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:22.756078 16002 webserver.cc:460] Webserver started at http://127.2.86.62:45525/ using document root <none> and password file <none>
I20260926 13:06:22.756338 16002 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:22.756389 16002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:22.756523 16002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:22.757733 16002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1"
format_stamp: "Formatted at 2026-09-26 13:06:22 on dist-test-slave-j4lq"
server_key: "2e548c75be759cf3323e76ce12a6acd1"
server_key_iv: "439990f1e4648e51247a705a449fa508"
server_key_version: "encryptionkey@0"
I20260926 13:06:22.758121 16002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1"
format_stamp: "Formatted at 2026-09-26 13:06:22 on dist-test-slave-j4lq"
server_key: "2e548c75be759cf3323e76ce12a6acd1"
server_key_iv: "439990f1e4648e51247a705a449fa508"
server_key_version: "encryptionkey@0"
I20260926 13:06:22.759531 16002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:06:22.760419 16017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:22.760675 16002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:22.760761 16002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1"
format_stamp: "Formatted at 2026-09-26 13:06:22 on dist-test-slave-j4lq"
server_key: "2e548c75be759cf3323e76ce12a6acd1"
server_key_iv: "439990f1e4648e51247a705a449fa508"
server_key_version: "encryptionkey@0"
I20260926 13:06:22.760829 16002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:22.782056 16002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:22.784703 16002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:22.784885 16002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:22.789134 16002 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40619
I20260926 13:06:22.789229 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40619 every 8 connection(s)
I20260926 13:06:22.789606 16002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:22.790532 16070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:22.792833 16070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1: Bootstrap starting.
I20260926 13:06:22.793467 16070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:22.793774 16070 log.cc:824] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:22.794569 16070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1: No bootstrap required, opened a new log
I20260926 13:06:22.794997 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16002
I20260926 13:06:22.795075 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:06:22.795296 2392 external_mini_cluster.cc:1428] Setting key 047ea65f945fb6d918145ce4388c86fb
I20260926 13:06:22.796736 16070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } }
I20260926 13:06:22.797005 16070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:22.797061 16070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf2a3d037a542e0b94f83ee20ac8bc1, State: Initialized, Role: FOLLOWER
I20260926 13:06:22.797245 16070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } }
I20260926 13:06:22.797354 16070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:22.797394 16070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:22.797480 16070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:22.798293 16070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } }
I20260926 13:06:22.798393 16070 leader_election.cc:302] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cf2a3d037a542e0b94f83ee20ac8bc1; no voters:
I20260926 13:06:22.798636 16070 leader_election.cc:288] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:06:22.798693 16075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:22.798846 16075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf2a3d037a542e0b94f83ee20ac8bc1, State: Running, Role: LEADER
I20260926 13:06:22.798943 16070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:22.798931 16075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } }
I20260926 13:06:22.799278 16075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cf2a3d037a542e0b94f83ee20ac8bc1. Latest consensus state: current_term: 1 leader_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } } }
I20260926 13:06:22.799361 16075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:22.799290 16077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf2a3d037a542e0b94f83ee20ac8bc1" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40619 } } }
I20260926 13:06:22.799398 16077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:22.800335 16079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:22.800987 16079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:22.802815 16079 catalog_manager.cc:1347] Generated new cluster ID: 5290739f33aa431f9ee63eaa4edd8453
I20260926 13:06:22.802870 16079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[15989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:22.807421 16079 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:06:22.808113 16079 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:22.808132 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:22.795834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.2.86.62:40619)
0926 13:06:22.796035 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:22.796043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:22.796068 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:06:22.796966 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:22.796982 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:22.797057 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:22.797241 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:22.798141 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:22.799570 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:22.800635 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:22.801269 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:22.801383 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:22.804274 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:22.804278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:22.804279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:22.804292 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:22.806207 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:22.806636 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:22.806639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:22.806639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:22.806734 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:22.806966 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:22.806966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:22.806967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:22.807269 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:22.807474 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:22.807842 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 13:06:22.807975 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260926 13:06:22.809104 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:06:22.816727 16079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cf2a3d037a542e0b94f83ee20ac8bc1: Generated new TSK 0
I20260926 13:06:22.816946 16079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:06:22.863443 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16002
2026-09-26T13:06:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1366 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[16097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[16097](info): set up 2 sockets
Sep 26 13:06:22 dist-test-slave-j4lq krb5kdc[16097](info): commencing operation
krb5kdc: starting...
W20260926 13:06:23.917038 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:06:23 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:06:24.040989 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:38481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:24.124289 16110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:24.124497 16110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:24.124521 16110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:24.125767 16110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:24.125809 16110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:24.125833 16110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:24.125846 16110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:24.125859 16110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:24.127269 16110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:38481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:24.127463 16110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:24.127643 16110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:24.132428 16116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.132452 16110 server_base.cc:1034] running on GCE node
W20260926 13:06:24.132416 16115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:24.132404 16118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.132848 16110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:24.133183 16110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:24.134344 16110 hybrid_clock.cc:648] HybridClock initialized: now 1790427984134335 us; error 30 us; skew 500 ppm
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:24.140673 16110 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:24.141384 16110 webserver.cc:460] Webserver started at http://127.2.86.62:35293/ using document root <none> and password file <none>
I20260926 13:06:24.141628 16110 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:24.141697 16110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:24.141852 16110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:24.142997 16110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "4a4e80cb46bd86baf091e2bd1d30267c"
server_key_iv: "71df5624e50165d17ec1613b78f00688"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.143381 16110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "4a4e80cb46bd86baf091e2bd1d30267c"
server_key_iv: "71df5624e50165d17ec1613b78f00688"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.144789 16110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:06:24.145645 16125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.145854 16110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:24.145993 16110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "4a4e80cb46bd86baf091e2bd1d30267c"
server_key_iv: "71df5624e50165d17ec1613b78f00688"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.146073 16110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:24.181291 16110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:24.183745 16110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:24.183913 16110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:24.188141 16110 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:38481
I20260926 13:06:24.188174 16177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:38481 every 8 connection(s)
I20260926 13:06:24.188529 16110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:24.189669 16178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:24.192296 16178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap starting.
I20260926 13:06:24.193008 16178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:24.193323 16178 log.cc:824] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:24.194119 16178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: No bootstrap required, opened a new log
I20260926 13:06:24.195830 16178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:24.195981 16178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:24.196008 16178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Initialized, Role: FOLLOWER
I20260926 13:06:24.196167 16178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:24.196229 16178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:24.196250 16178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:24.196308 16178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:24.196310 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16110
I20260926 13:06:24.196448 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:06:24.196635 2392 external_mini_cluster.cc:1428] Setting key 6064aae16c97ac90dabbc897371a0c56
I20260926 13:06:24.197460 16178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:24.197589 16178 leader_election.cc:302] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7709ae46f6fc4127ad3a0a5ee17eafd2; no voters:
I20260926 13:06:24.197746 16178 leader_election.cc:288] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:06:24.197774 16183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:24.197994 16183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 LEADER]: Becoming Leader. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Running, Role: LEADER
I20260926 13:06:24.198022 16178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:24.198204 16183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:24.199222 16184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7709ae46f6fc4127ad3a0a5ee17eafd2. Latest consensus state: current_term: 1 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:24.199347 16184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:24.199791 16185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:24.200637 16185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
W20260926 13:06:24.200098 16199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:06:24.200697 16199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:06:24.200757 16197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:24.200932 16197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:24.203449 16197 catalog_manager.cc:1347] Generated new cluster ID: 2a878b40e3394eb5855a96f2f03cbd58
I20260926 13:06:24.203513 16197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:24.208328 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.197243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.2.86.62:38481)
0926 13:06:24.197435 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.197446 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.197482 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:06:24.197611 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.197619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.197677 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.197822 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:24.198611 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.200213 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.201342 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.201837 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.201990 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.204659 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:24.204663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:24.204663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:24.204675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:24.206673 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.207096 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.207099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.207099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.207212 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.207418 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.207419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.207419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.207722 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:24.207866 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.208001 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.208131 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260926 13:06:24.214779 16197 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:06:24.215559 16197 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:24.222656 16197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Generated new TSK 0
I20260926 13:06:24.222860 16197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:06:24.257936 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:24.341609 16204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:24.341833 16204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:24.341859 16204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:24.343050 16204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:24.343093 16204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:24.343122 16204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:24.344519 16204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:24.344720 16204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:24.344969 16204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:06:24.350188 16204 server_base.cc:1034] running on GCE node
W20260926 13:06:24.350153 16212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:24.350149 16210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:24.350139 16209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.350595 16204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:24.350867 16204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:24.352031 16204 hybrid_clock.cc:648] HybridClock initialized: now 1790427984352026 us; error 41 us; skew 500 ppm
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:24.354650 16204 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:24.355108 16204 webserver.cc:460] Webserver started at http://127.2.86.1:46859/ using document root <none> and password file <none>
I20260926 13:06:24.355337 16204 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:24.355403 16204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:24.355561 16204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:24.356627 16204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "c89cc71e8e894a98a134dc16c126b6de"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "a48b1213ac5f1c5cc1e8a65a7e3dd0bd"
server_key_iv: "ce34395acbeda60531932549281fe3b9"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.357028 16204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "c89cc71e8e894a98a134dc16c126b6de"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "a48b1213ac5f1c5cc1e8a65a7e3dd0bd"
server_key_iv: "ce34395acbeda60531932549281fe3b9"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.358353 16204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:24.359228 16219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.359431 16204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:24.359549 16204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c89cc71e8e894a98a134dc16c126b6de"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "a48b1213ac5f1c5cc1e8a65a7e3dd0bd"
server_key_iv: "ce34395acbeda60531932549281fe3b9"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.359622 16204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:24.367710 16204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:24.370005 16204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:24.370168 16204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:24.370406 16204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:24.370733 16204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:24.370784 16204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.370826 16204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:24.370853 16204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.376621 16204 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41747
I20260926 13:06:24.376694 16332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41747 every 8 connection(s)
I20260926 13:06:24.377075 16204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:06:24.382946 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16204
I20260926 13:06:24.383086 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:06:24.383306 2392 external_mini_cluster.cc:1428] Setting key 8ea1383986753676ebc28c705417fa97
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:24.387276 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.377827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:52235)
0926 13:06:24.377998 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.378033 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.378560 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.378818 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.378823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.379448 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:24.379648 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.379653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.380377 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.380378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.381252 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.381255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.381650 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.381651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.381765 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.382315 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:24.382323 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:24.383876 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:24.385926 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.385933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.385935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.386166 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.386371 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.386372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.386372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.386470 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.386774 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:24.386778 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:24.386899 (+ 121us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.386979 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.387106 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260926 13:06:24.387269 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.377849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:52235 (local address 127.2.86.62:38481)
0926 13:06:24.377894 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.377896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.378138 (+ 242us) server_negotiation.cc:407] Connection header received
0926 13:06:24.378672 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.378673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.378685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.378742 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:24.379749 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.380266 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.381337 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.381546 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.381637 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.383984 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:24.383986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:24.383986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:24.383995 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:24.385841 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.386243 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.386244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.386245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.386306 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.386562 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.386563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.386563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.386712 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:24.386781 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.387022 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.387127 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:06:24.387841 16333 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:24.387949 16333 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:24.388136 16333 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:24.388693 16142 ts_manager.cc:194] Registered new tserver with Master: c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747)
I20260926 13:06:24.389429 16142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:52235
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:06:24.412252 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:24.494386 16338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:24.494660 16338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:24.494702 16338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:24.495926 16338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:24.495999 16338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:24.496045 16338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:24.497689 16338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:24.497977 16338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:24.498245 16338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:24.504283 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.504285 16338 server_base.cc:1034] running on GCE node
W20260926 13:06:24.504354 16346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:24.504303 16343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.504667 16338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:24.505007 16338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:24.506182 16338 hybrid_clock.cc:648] HybridClock initialized: now 1790427984506175 us; error 59 us; skew 500 ppm
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:24.509047 16338 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:24.509457 16338 webserver.cc:460] Webserver started at http://127.2.86.2:46185/ using document root <none> and password file <none>
I20260926 13:06:24.509665 16338 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:24.509708 16338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:24.509800 16338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:24.510847 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "173852ee80a0430289e2cddfee8b1428"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "07b7072a0843b3141825a58fb877089f"
server_key_iv: "6c74abe988233afa9feb8cafd5e8bb8f"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.511162 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "173852ee80a0430289e2cddfee8b1428"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "07b7072a0843b3141825a58fb877089f"
server_key_iv: "6c74abe988233afa9feb8cafd5e8bb8f"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.512449 16338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:06:24.513315 16353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.513504 16338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:06:24.513586 16338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "173852ee80a0430289e2cddfee8b1428"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "07b7072a0843b3141825a58fb877089f"
server_key_iv: "6c74abe988233afa9feb8cafd5e8bb8f"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.513643 16338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:24.530390 16338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:24.532330 16338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:24.532436 16338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:24.532631 16338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:24.533015 16338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:24.533051 16338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.533075 16338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:24.533088 16338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.538691 16338 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:33983
I20260926 13:06:24.538784 16466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:33983 every 8 connection(s)
I20260926 13:06:24.539083 16338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:24.547448 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16338
I20260926 13:06:24.547575 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:06:24.547793 2392 external_mini_cluster.cc:1428] Setting key 2d9d2d002269993e320f8fa5925d22b5
I20260926 13:06:24.548528 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.539791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:54971 (local address 127.2.86.62:38481)
0926 13:06:24.539836 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.539839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.540014 (+ 175us) server_negotiation.cc:407] Connection header received
0926 13:06:24.540524 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.540526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.540537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.540588 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:24.541601 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.542121 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.543105 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.543307 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.543379 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.545498 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:24.545499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:24.545500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:24.545508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:24.546890 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.547290 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.547291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.547292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.547351 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.547613 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.547614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.547615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.547786 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:24.547885 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.548234 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.548375 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:06:24.548694 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.539760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:54971)
0926 13:06:24.539889 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.539934 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.540423 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.540657 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.540661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.541295 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:24.541485 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.541490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.542214 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.542215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.543003 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.543005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.543382 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.543383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.543511 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.544022 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:24.544030 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:24.545404 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:24.546988 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.546995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.546997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.547217 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.547414 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.547415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.547416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.547531 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.547875 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:24.547881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:24.548027 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.548184 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.548368 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260926 13:06:24.549183 16467 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:24.549278 16467 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:24.549485 16467 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:24.549911 16142 ts_manager.cc:194] Registered new tserver with Master: 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
I20260926 13:06:24.550412 16142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:54971
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:06:24.575999 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:24.656656 16472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:24.656932 16472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:24.656981 16472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:24.658167 16472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:24.658236 16472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:24.658280 16472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:24.659734 16472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:24.659988 16472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:24.660213 16472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:24.665141 16478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.665246 16472 server_base.cc:1034] running on GCE node
W20260926 13:06:24.665149 16477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:24.665149 16480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:24.665617 16472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:24.665890 16472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:24.667042 16472 hybrid_clock.cc:648] HybridClock initialized: now 1790427984667025 us; error 35 us; skew 500 ppm
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:24.669622 16472 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:24.670043 16472 webserver.cc:460] Webserver started at http://127.2.86.3:39295/ using document root <none> and password file <none>
I20260926 13:06:24.670240 16472 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:24.670280 16472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:24.670365 16472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:24.671370 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "ee14d479544749cc84f206d7461c4833"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "3ea768fdece359773141398ece91c9a1"
server_key_iv: "0ce318b7a8636ca88330c171c597c0b1"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.671716 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "ee14d479544749cc84f206d7461c4833"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "3ea768fdece359773141398ece91c9a1"
server_key_iv: "0ce318b7a8636ca88330c171c597c0b1"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.673168 16472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:06:24.673960 16487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.674250 16472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:24.674343 16472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "ee14d479544749cc84f206d7461c4833"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "3ea768fdece359773141398ece91c9a1"
server_key_iv: "0ce318b7a8636ca88330c171c597c0b1"
server_key_version: "encryptionkey@0"
I20260926 13:06:24.674419 16472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:24.695227 16472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:24.697352 16472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:24.697506 16472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:24.697716 16472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:24.698105 16472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:24.698149 16472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.698173 16472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:24.698187 16472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:24.704591 16472 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:33663
I20260926 13:06:24.704681 16600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:33663 every 8 connection(s)
I20260926 13:06:24.705083 16472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:06:24.711050 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16472
I20260926 13:06:24.711155 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
Sep 26 13:06:24 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:24.711328 2392 external_mini_cluster.cc:1428] Setting key 148d42d7c6c9735d1b6b13a4e4bbe38b
I20260926 13:06:24.714358 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.705867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:34029 (local address 127.2.86.62:38481)
0926 13:06:24.705936 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.705939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.706169 (+ 230us) server_negotiation.cc:407] Connection header received
0926 13:06:24.706656 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.706657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.706667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.706711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:24.707682 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.708183 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.709283 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.709470 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.709528 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.711695 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:24.711696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:24.711696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:24.711703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:24.713133 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.713520 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.713521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.713521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.713571 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.713842 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.713843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.713843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.713941 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:24.713990 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.714234 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.714294 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:06:24.714563 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.705838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:34029)
0926 13:06:24.706052 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.706089 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.706560 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.706790 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.706793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.707403 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:24.707604 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.707609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.708288 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.708289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.709207 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.709210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.709544 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.709544 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.709642 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.710154 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:24.710162 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:24.711599 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:24.713204 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.713210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.713212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.713448 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.713646 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:24.713647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:24.713647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:24.713781 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:24.713995 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:24.714000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:24.714122 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.714210 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.714336 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260926 13:06:24.715045 16601 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:24.715165 16601 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:24.715356 16601 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:24.715741 16142 ts_manager.cc:194] Registered new tserver with Master: ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663)
I20260926 13:06:24.716110 16142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:34029
I20260926 13:06:24.723184 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:06:24.730216 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.724514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.2.86.62:38481)
0926 13:06:24.724563 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.724565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.724696 (+ 131us) server_negotiation.cc:407] Connection header received
0926 13:06:24.724743 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.724743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.724752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.724801 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:24.725415 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.725883 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.726581 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.726753 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.726801 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.727950 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:24.727951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:24.727951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:24.727956 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:24.729295 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.729608 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.729608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.729609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.729657 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:24.729806 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:24.729806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:24.729806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:24.729937 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:24.730008 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.730093 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.730150 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:06:24.731916 16142 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:06:24.733614 16142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:06:24.740123 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.736306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.2.86.2:33983)
0926 13:06:24.736476 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.736481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.736492 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:06:24.736539 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.736541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.736555 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.736629 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:24.736831 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.737898 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.739039 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.739736 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.739813 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.739873 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.739971 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.740020 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260926 13:06:24.740118 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.736099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:33983 (local address 127.0.0.1:55550)
0926 13:06:24.736292 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.736302 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.736338 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.736706 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.736707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.736708 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:24.736751 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.736753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.738004 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.738006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.738945 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.738947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.739819 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.739820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.739877 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.739890 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.739945 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.739984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":55,"threads_started":1}
I20260926 13:06:24.740293 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.735897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41747 (local address 127.0.0.1:47030)
0926 13:06:24.736044 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.736085 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.736148 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.736458 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.736461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.736463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:24.736529 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.736530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.737899 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.737900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.739171 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.739174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.740051 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.740052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.740118 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.740122 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.740175 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.740221 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":77,"thread_start_us":46,"threads_started":1}
I20260926 13:06:24.740552 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.735997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.2.86.1:41747)
0926 13:06:24.736217 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.736223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.736232 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:06:24.736274 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.736277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.736293 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.736369 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:24.736609 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.737790 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.739258 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.739969 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.740041 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.740093 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.740400 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.740447 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
I20260926 13:06:24.741016 16401 tablet_service.cc:1467] Processing CreateTablet for tablet 2970817de09540fcacaf5e0f7ea54333 (DEFAULT_TABLE table=test-table [id=5c73c2929206445da5f2e4f95b026ed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:24.741236 16267 tablet_service.cc:1467] Processing CreateTablet for tablet 2970817de09540fcacaf5e0f7ea54333 (DEFAULT_TABLE table=test-table [id=5c73c2929206445da5f2e4f95b026ed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:24.741426 16401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2970817de09540fcacaf5e0f7ea54333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:24.741593 16267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2970817de09540fcacaf5e0f7ea54333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:24.741917 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.735898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33663 (local address 127.0.0.1:58816)
0926 13:06:24.736192 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.736204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.736256 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.736654 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.736654 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.736656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:24.736707 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.736709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.737775 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.737776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.739174 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.739175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.741648 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.741649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.741722 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.741726 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.741793 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.741837 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":56,"thread_start_us":79,"threads_started":1}
I20260926 13:06:24.741940 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.735995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.2.86.3:33663)
0926 13:06:24.736476 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.736481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.736491 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:06:24.736532 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.736534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.736550 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.736932 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:24.737009 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.737678 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.740133 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.741559 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.741648 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.741718 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.741809 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.741846 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":90,"threads_started":1}
I20260926 13:06:24.742575 16535 tablet_service.cc:1467] Processing CreateTablet for tablet 2970817de09540fcacaf5e0f7ea54333 (DEFAULT_TABLE table=test-table [id=5c73c2929206445da5f2e4f95b026ed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:24.742925 16535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2970817de09540fcacaf5e0f7ea54333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:24.744632 16618 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Bootstrap starting.
I20260926 13:06:24.745118 16619 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap starting.
I20260926 13:06:24.745410 16618 tablet_bootstrap.cc:654] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:24.745918 16619 tablet_bootstrap.cc:654] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:24.746049 16618 log.cc:824] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:24.746532 16619 log.cc:824] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:24.746970 16618 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: No bootstrap required, opened a new log
I20260926 13:06:24.747057 16618 ts_tablet_manager.cc:1397] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:06:24.747287 16619 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: No bootstrap required, opened a new log
I20260926 13:06:24.747350 16619 ts_tablet_manager.cc:1397] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:24.748780 16619 raft_consensus.cc:348] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.748800 16618 raft_consensus.cc:348] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.748983 16619 raft_consensus.cc:374] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:24.748992 16618 raft_consensus.cc:374] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:24.749025 16619 raft_consensus.cc:729] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89cc71e8e894a98a134dc16c126b6de, State: Initialized, Role: FOLLOWER
I20260926 13:06:24.749042 16618 raft_consensus.cc:729] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173852ee80a0430289e2cddfee8b1428, State: Initialized, Role: FOLLOWER
I20260926 13:06:24.749063 16621 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Bootstrap starting.
I20260926 13:06:24.749188 16618 consensus_queue.cc:260] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.749187 16619 consensus_queue.cc:260] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.749472 16619 ts_tablet_manager.cc:1428] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:24.749516 16618 ts_tablet_manager.cc:1428] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260926 13:06:24.749624 16467 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
I20260926 13:06:24.749627 16333 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
I20260926 13:06:24.749854 16621 tablet_bootstrap.cc:654] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:24.750453 16621 log.cc:824] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:24.751235 16621 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: No bootstrap required, opened a new log
I20260926 13:06:24.751291 16621 ts_tablet_manager.cc:1397] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:24.752810 16621 raft_consensus.cc:348] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.753041 16621 raft_consensus.cc:374] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:24.753068 16621 raft_consensus.cc:729] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee14d479544749cc84f206d7461c4833, State: Initialized, Role: FOLLOWER
I20260926 13:06:24.753180 16621 consensus_queue.cc:260] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.753414 16621 ts_tablet_manager.cc:1428] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:24.753464 16601 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
W20260926 13:06:24.789675 16468 tablet.cc:2369] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:06:24.877875 16334 tablet.cc:2369] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:06:24.897941 16624 raft_consensus.cc:482] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:24.898079 16624 raft_consensus.cc:504] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.898564 16624 leader_election.cc:288] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
I20260926 13:06:24.901755 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.899010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:50657 (local address 127.2.86.2:33983)
0926 13:06:24.899066 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.899068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.899098 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:06:24.899174 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.899176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.899188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.899241 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:24.899430 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.899884 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901005 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.901398 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901482 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.901492 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.901642 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.901679 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 13:06:24.901764 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.898841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:54765 (local address 127.2.86.3:33663)
0926 13:06:24.898883 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.898885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.898934 (+ 49us) server_negotiation.cc:407] Connection header received
0926 13:06:24.899075 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.899077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.899095 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.899161 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:24.899353 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.899865 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901027 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.901401 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901483 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.901500 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.901652 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.901694 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:06:24.901798 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.898933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:33983 (local address 127.2.86.1:50657)
0926 13:06:24.899026 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.899038 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.899119 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.899249 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.899250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.899252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:24.899341 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.899343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.899967 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.899968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.900901 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.900904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.901483 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.901566 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.901574 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.901637 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.901679 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":62,"thread_start_us":52,"threads_started":1}
I20260926 13:06:24.901795 16627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.898743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33663 (local address 127.2.86.1:54765)
0926 13:06:24.898870 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.898882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:24.898953 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:24.899166 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:24.899167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:24.899169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:24.899244 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.899246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.899950 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.899951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.900935 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:24.900938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.901486 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.901487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:24.901562 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.901566 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:24.901644 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:06:24.901689 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260926 13:06:24.902233 16555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2970817de09540fcacaf5e0f7ea54333" candidate_uuid: "c89cc71e8e894a98a134dc16c126b6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee14d479544749cc84f206d7461c4833" is_pre_election: true
I20260926 13:06:24.902273 16421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2970817de09540fcacaf5e0f7ea54333" candidate_uuid: "c89cc71e8e894a98a134dc16c126b6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173852ee80a0430289e2cddfee8b1428" is_pre_election: true
I20260926 13:06:24.902384 16555 raft_consensus.cc:2454] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c89cc71e8e894a98a134dc16c126b6de in term 0.
I20260926 13:06:24.902436 16421 raft_consensus.cc:2454] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c89cc71e8e894a98a134dc16c126b6de in term 0.
I20260926 13:06:24.902626 16221 leader_election.cc:302] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de, ee14d479544749cc84f206d7461c4833; no voters:
I20260926 13:06:24.902758 16624 raft_consensus.cc:2781] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:06:24.902839 16624 raft_consensus.cc:482] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:06:24.902873 16624 raft_consensus.cc:3037] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:24.903656 16624 raft_consensus.cc:504] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.903810 16624 leader_election.cc:288] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 1 election: Requested vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
I20260926 13:06:24.904088 16421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2970817de09540fcacaf5e0f7ea54333" candidate_uuid: "c89cc71e8e894a98a134dc16c126b6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173852ee80a0430289e2cddfee8b1428"
I20260926 13:06:24.904079 16555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2970817de09540fcacaf5e0f7ea54333" candidate_uuid: "c89cc71e8e894a98a134dc16c126b6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee14d479544749cc84f206d7461c4833"
I20260926 13:06:24.904170 16555 raft_consensus.cc:3037] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:24.904168 16421 raft_consensus.cc:3037] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:24.905136 16555 raft_consensus.cc:2454] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c89cc71e8e894a98a134dc16c126b6de in term 1.
I20260926 13:06:24.905136 16421 raft_consensus.cc:2454] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c89cc71e8e894a98a134dc16c126b6de in term 1.
I20260926 13:06:24.905296 16221 leader_election.cc:302] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de, ee14d479544749cc84f206d7461c4833; no voters:
I20260926 13:06:24.905406 16624 raft_consensus.cc:2781] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:24.905612 16624 raft_consensus.cc:686] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 LEADER]: Becoming Leader. State: Replica: c89cc71e8e894a98a134dc16c126b6de, State: Running, Role: LEADER
I20260926 13:06:24.905725 16624 consensus_queue.cc:237] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:24.906582 16141 catalog_manager.cc:5495] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de reported cstate change: term changed from 0 to 1, leader changed from <none> to c89cc71e8e894a98a134dc16c126b6de (127.2.86.1). New cstate: current_term: 1 leader_uuid: "c89cc71e8e894a98a134dc16c126b6de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260926 13:06:24.921505 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.919142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.2.86.1:41747)
0926 13:06:24.919198 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:24.919200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:24.919214 (+ 14us) server_negotiation.cc:407] Connection header received
0926 13:06:24.919248 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:24.919249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:24.919267 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:24.919328 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:06:24.919500 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.919960 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.920719 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:24.920942 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:24.920993 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:24.921163 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:06:24.921261 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:06:24.921317 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:24.921385 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 13:06:24.921403 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:06:24.923342 16555 raft_consensus.cc:1261] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Refusing update from remote peer c89cc71e8e894a98a134dc16c126b6de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:24.923363 16421 raft_consensus.cc:1261] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Refusing update from remote peer c89cc71e8e894a98a134dc16c126b6de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:24.923731 16629 consensus_queue.cc:1035] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:24.923868 16624 consensus_queue.cc:1035] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [LEADER]: Connected to new peer: Peer: permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:24.926855 16633 mvcc.cc:205] Tried to move back new op lower bound from 7333593026243366912 to 7333593026174521344. Current Snapshot: MvccSnapshot[applied={T|T < 7333593026243366912}]
I20260926 13:06:24.926965 16632 mvcc.cc:205] Tried to move back new op lower bound from 7333593026243366912 to 7333593026174521344. Current Snapshot: MvccSnapshot[applied={T|T < 7333593026243366912 or (T in {7333593026243366912})}]
I20260926 13:06:24.927814 16634 mvcc.cc:205] Tried to move back new op lower bound from 7333593026243366912 to 7333593026174521344. Current Snapshot: MvccSnapshot[applied={T|T < 7333593026243366912}]
I20260926 13:06:24.928937 16141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:24.929042 16141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:24.929792 16141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Sending DeleteTablet for 3 replicas of tablet 2970817de09540fcacaf5e0f7ea54333
I20260926 13:06:24.930099 16267 tablet_service.cc:1514] Processing DeleteTablet for tablet 2970817de09540fcacaf5e0f7ea54333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:24 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:47030
I20260926 13:06:24.930152 16401 tablet_service.cc:1514] Processing DeleteTablet for tablet 2970817de09540fcacaf5e0f7ea54333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:24 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:55550
I20260926 13:06:24.930184 16535 tablet_service.cc:1514] Processing DeleteTablet for tablet 2970817de09540fcacaf5e0f7ea54333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:24 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:58816
I20260926 13:06:24.930388 16643 tablet_replica.cc:330] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: stopping tablet replica
I20260926 13:06:24.930444 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16204
I20260926 13:06:24.930496 16643 raft_consensus.cc:2229] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:24.930611 16643 raft_consensus.cc:2258] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:24.931001 16642 tablet_replica.cc:330] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: stopping tablet replica
I20260926 13:06:24.931066 16643 ts_tablet_manager.cc:1905] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:24.931092 16642 raft_consensus.cc:2229] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:24.931181 16642 raft_consensus.cc:2258] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:24.931614 16642 ts_tablet_manager.cc:1905] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:24.933251 16643 ts_tablet_manager.cc:1918] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:24.933323 16643 log.cc:1196] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/2970817de09540fcacaf5e0f7ea54333
I20260926 13:06:24.933399 16642 ts_tablet_manager.cc:1918] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:24.933459 16642 log.cc:1196] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/2970817de09540fcacaf5e0f7ea54333
I20260926 13:06:24.933624 16643 ts_tablet_manager.cc:1939] T 2970817de09540fcacaf5e0f7ea54333 P ee14d479544749cc84f206d7461c4833: Deleting consensus metadata
I20260926 13:06:24.933719 16642 ts_tablet_manager.cc:1939] T 2970817de09540fcacaf5e0f7ea54333 P 173852ee80a0430289e2cddfee8b1428: Deleting consensus metadata
I20260926 13:06:24.934079 16127 catalog_manager.cc:4841] TS ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): tablet 2970817de09540fcacaf5e0f7ea54333 (table test-table [id=5c73c2929206445da5f2e4f95b026ed0]) successfully deleted
I20260926 13:06:24.934191 16127 catalog_manager.cc:4841] TS 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): tablet 2970817de09540fcacaf5e0f7ea54333 (table test-table [id=5c73c2929206445da5f2e4f95b026ed0]) successfully deleted
W20260926 13:06:24.935741 16128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:41747 (error 108)
I20260926 13:06:24.936076 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16338
I20260926 13:06:24.936672 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:24.936128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41747 (local address 127.0.0.1:47044)
0926 13:06:24.936598 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:24.936644 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451}
W20260926 13:06:24.936813 16128 catalog_manager.cc:4584] TS c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2970817de09540fcacaf5e0f7ea54333: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41747: connect: Connection refused (error 111)
I20260926 13:06:24.940943 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16472
I20260926 13:06:24.945794 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16110
I20260926 13:06:24.951128 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--webserver_interface=127.2.86.62
--webserver_port=35293
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:38481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:25.037297 16645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.037541 16645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.037578 16645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.038712 16645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:25.038776 16645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.038803 16645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.038833 16645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:25.038857 16645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:25.041021 16645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:38481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.041301 16645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.041534 16645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:25.050410 16652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.050415 16651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.050407 16654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.050675 16645 server_base.cc:1034] running on GCE node
I20260926 13:06:25.050976 16645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.051306 16645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.052500 16645 hybrid_clock.cc:648] HybridClock initialized: now 1790427985052488 us; error 35 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.055326 16645 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:25.055912 16645 webserver.cc:460] Webserver started at http://127.2.86.62:35293/ using document root <none> and password file <none>
I20260926 13:06:25.056181 16645 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.056269 16645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.057883 16645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.058653 16661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.058828 16645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.058933 16645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "4a4e80cb46bd86baf091e2bd1d30267c"
server_key_iv: "71df5624e50165d17ec1613b78f00688"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.059369 16645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.073655 16645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.075973 16645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.076148 16645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.080428 16645 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:38481
I20260926 13:06:25.080461 16713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:38481 every 8 connection(s)
I20260926 13:06:25.080916 16645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:25.082602 16714 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:06:25.083292 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap starting.
I20260926 13:06:25.086231 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16645
I20260926 13:06:25.086567 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:41747
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=46859
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:25.092044 16714 log.cc:824] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.094035 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:06:25.094331 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap complete.
I20260926 13:06:25.096654 16714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.096942 16714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.097126 16714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.097188 16714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:25.097250 16714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:25.097302 16714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:06:25.098330 16714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.098439 16714 leader_election.cc:302] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7709ae46f6fc4127ad3a0a5ee17eafd2; no voters:
I20260926 13:06:25.098640 16714 leader_election.cc:288] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:06:25.098680 16718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:06:25.098793 16718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 LEADER]: Becoming Leader. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Running, Role: LEADER
I20260926 13:06:25.098881 16718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.098918 16714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:25.099143 16718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:25.099198 16718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:25.099323 16718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7709ae46f6fc4127ad3a0a5ee17eafd2. Latest consensus state: current_term: 2 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:25.099385 16718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:25.099469 16722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:25.100296 16722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:25.100437 16722 catalog_manager.cc:1259] Loaded cluster ID: 2a878b40e3394eb5855a96f2f03cbd58
I20260926 13:06:25.100688 16722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:06:25.101003 16722 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 13:06:25.105646 16722 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 13:06:25.108217 16722 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:25.108426 16722 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Loaded TSK: 0
I20260926 13:06:25.108654 16722 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:06:25.171365 16716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.171617 16716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.171659 16716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.172978 16716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.173040 16716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.173087 16716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:25.174659 16716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:41747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=46859
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.174954 16716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.175226 16716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:25.181289 16742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.181558 16716 server_base.cc:1034] running on GCE node
W20260926 13:06:25.181268 16741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.181288 16744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.181869 16716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.182170 16716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.183353 16716 hybrid_clock.cc:648] HybridClock initialized: now 1790427985183331 us; error 37 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.186591 16716 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:25.187115 16716 webserver.cc:460] Webserver started at http://127.2.86.1:46859/ using document root <none> and password file <none>
I20260926 13:06:25.187394 16716 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.187453 16716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.188964 16716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:06:25.189678 16751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.189927 16716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.190061 16716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c89cc71e8e894a98a134dc16c126b6de"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "a48b1213ac5f1c5cc1e8a65a7e3dd0bd"
server_key_iv: "ce34395acbeda60531932549281fe3b9"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.190479 16716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.205868 16716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.208447 16716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.208633 16716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.208940 16716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:25.209414 16758 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:06:25.211354 16716 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:06:25.211437 16716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 13:06:25.211489 16716 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:06:25.212141 16716 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:06:25.212193 16716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 13:06:25.212267 16758 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap starting.
I20260926 13:06:25.219766 16716 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41747
I20260926 13:06:25.220284 16716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:06:25.221045 16865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41747 every 8 connection(s)
I20260926 13:06:25.222024 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16716
I20260926 13:06:25.222321 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:33983
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=46185
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:25.223726 16758 log.cc:824] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.226701 16758 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:06:25.227094 16758 tablet_bootstrap.cc:492] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap complete.
I20260926 13:06:25.228348 16758 ts_tablet_manager.cc:1397] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260926 13:06:25.229832 16758 raft_consensus.cc:348] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:25.230156 16758 raft_consensus.cc:729] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89cc71e8e894a98a134dc16c126b6de, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.230297 16758 consensus_queue.cc:260] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } } peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } }
I20260926 13:06:25.230605 16758 ts_tablet_manager.cc:1428] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Time spent starting tablet: real 0.002s user 0.001s sys 0.004s
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:25.236791 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.221123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:36215)
0926 13:06:25.224038 (+ 2915us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.224086 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.225472 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.226173 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.226178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.226533 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:25.226991 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.226999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.228920 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.228921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.230297 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.230299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.231199 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.231200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.231293 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.231867 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:25.231876 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:25.233260 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:25.235187 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.235195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.235197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.235427 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.235745 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.235746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.235747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.235855 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.236240 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:25.236246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:25.236384 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.236504 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.236643 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2860,"thread_start_us":108,"threads_started":1}
I20260926 13:06:25.237444 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.221552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:36215 (local address 127.2.86.62:38481)
0926 13:06:25.221760 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.221770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.224228 (+ 2458us) server_negotiation.cc:407] Connection header received
0926 13:06:25.225732 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.225741 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.225802 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.225917 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:25.227145 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.228690 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.230458 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.230966 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.231037 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.233371 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:25.233375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:25.233375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:25.233388 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:25.235067 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.235544 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.235547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.235547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.235670 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.235922 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.235923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.235923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.236161 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:25.236249 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.237136 (+ 887us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.237291 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260926 13:06:25.238103 16866 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:25.238200 16866 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:25.238386 16866 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:25.239091 16678 ts_manager.cc:194] Registered new tserver with Master: c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747)
I20260926 13:06:25.240217 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:36215
I20260926 13:06:25.241196 16866 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
I20260926 13:06:25.244122 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.239683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41747 (local address 127.0.0.1:47056)
0926 13:06:25.240982 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.241021 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.241093 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.241380 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.241383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.241384 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:25.241448 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.241450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.242350 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.242351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.243153 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.243155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.243803 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.243804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.243883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.243895 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.243996 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.244051 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1268,"thread_start_us":96,"threads_started":1}
I20260926 13:06:25.244122 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.239936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.2.86.1:41747)
0926 13:06:25.240109 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.240114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.241173 (+ 1059us) server_negotiation.cc:407] Connection header received
0926 13:06:25.241219 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.241222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.241237 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.241306 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:25.241536 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.242262 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.243241 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.243719 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.243803 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.243858 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.244013 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.244051 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260926 13:06:25.244769 16782 tablet_service.cc:1514] Processing DeleteTablet for tablet 2970817de09540fcacaf5e0f7ea54333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:24 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:47056
I20260926 13:06:25.245339 16875 tablet_replica.cc:330] stopping tablet replica
I20260926 13:06:25.245460 16875 raft_consensus.cc:2229] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:25.245529 16875 raft_consensus.cc:2258] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:25.245826 16875 ts_tablet_manager.cc:1905] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:25.248153 16875 ts_tablet_manager.cc:1918] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:25.248239 16875 log.cc:1196] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/2970817de09540fcacaf5e0f7ea54333
I20260926 13:06:25.248452 16875 ts_tablet_manager.cc:1939] T 2970817de09540fcacaf5e0f7ea54333 P c89cc71e8e894a98a134dc16c126b6de: Deleting consensus metadata
I20260926 13:06:25.248965 16664 catalog_manager.cc:4841] TS c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747): tablet 2970817de09540fcacaf5e0f7ea54333 (table test-table [id=5c73c2929206445da5f2e4f95b026ed0]) successfully deleted
W20260926 13:06:25.312227 16870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.312445 16870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.312471 16870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.313758 16870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.313809 16870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.313838 16870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:25.315330 16870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:33983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=46185
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.315577 16870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.315834 16870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:25.321010 16880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.321077 16870 server_base.cc:1034] running on GCE node
W20260926 13:06:25.321029 16881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.321038 16883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.321391 16870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.321604 16870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.322748 16870 hybrid_clock.cc:648] HybridClock initialized: now 1790427985322727 us; error 40 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.325484 16870 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:25.325948 16870 webserver.cc:460] Webserver started at http://127.2.86.2:46185/ using document root <none> and password file <none>
I20260926 13:06:25.326174 16870 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.326247 16870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.327589 16870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.328235 16890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.328418 16870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:06:25.328531 16870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "173852ee80a0430289e2cddfee8b1428"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "07b7072a0843b3141825a58fb877089f"
server_key_iv: "6c74abe988233afa9feb8cafd5e8bb8f"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.329067 16870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.339920 16870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.342221 16870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.342334 16870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.342523 16870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:25.342823 16870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:25.342856 16870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.342897 16870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:25.342923 16870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.348820 16870 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:33983
I20260926 13:06:25.348917 17003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:33983 every 8 connection(s)
I20260926 13:06:25.349269 16870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:25.358292 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 16870
I20260926 13:06:25.358621 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:33663
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=39295
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:25.358853 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.350015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:35039 (local address 127.2.86.62:38481)
0926 13:06:25.350089 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.350091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.350301 (+ 210us) server_negotiation.cc:407] Connection header received
0926 13:06:25.350830 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.350831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.350843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.350895 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:25.351845 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.352351 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.353422 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.353630 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.353713 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.355885 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:25.355886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:25.355887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:25.355924 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:25.357380 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.357774 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.357775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.357776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.357831 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.358087 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.358088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.358088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.358222 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:25.358279 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.358627 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.358736 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 13:06:25.359112 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.350019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:35039)
0926 13:06:25.350185 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.350222 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.350729 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.350976 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.350985 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.351558 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:25.351747 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.351752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.352451 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.352451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.353335 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.353337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.353722 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.353723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.353893 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.354360 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:25.354368 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:25.355769 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:25.357457 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.357463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.357465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.357704 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.357889 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.357890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.357890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.357985 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.358289 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:25.358295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:25.358475 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.358610 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.358799 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260926 13:06:25.359640 17004 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:25.359741 17004 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:25.359953 17004 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:25.360391 16678 ts_manager.cc:194] Registered new tserver with Master: 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
I20260926 13:06:25.360764 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:35039
W20260926 13:06:25.441228 17007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.441491 17007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.441534 17007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.442768 17007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.442839 17007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.442880 17007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:25.444581 17007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=39295
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.444895 17007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.445182 17007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:06:25.450354 17007 server_base.cc:1034] running on GCE node
W20260926 13:06:25.450368 17013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.450366 17012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.450506 17015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.450769 17007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.451018 17007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.452194 17007 hybrid_clock.cc:648] HybridClock initialized: now 1790427985452177 us; error 55 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.454861 17007 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:25.455294 17007 webserver.cc:460] Webserver started at http://127.2.86.3:39295/ using document root <none> and password file <none>
I20260926 13:06:25.455480 17007 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.455533 17007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.456830 17007 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.457530 17022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.457695 17007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:06:25.457793 17007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "ee14d479544749cc84f206d7461c4833"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "3ea768fdece359773141398ece91c9a1"
server_key_iv: "0ce318b7a8636ca88330c171c597c0b1"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.458182 17007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.466087 17007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.468469 17007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.468633 17007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.468914 17007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:25.469262 17007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:25.469298 17007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.469322 17007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:25.469336 17007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.475134 17007 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:33663
I20260926 13:06:25.475189 17135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:33663 every 8 connection(s)
I20260926 13:06:25.475553 17007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:25.483903 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17007
I20260926 13:06:25.485658 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.476324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:36829 (local address 127.2.86.62:38481)
0926 13:06:25.476394 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.476397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.476623 (+ 226us) server_negotiation.cc:407] Connection header received
0926 13:06:25.477224 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.477225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.477236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.477290 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:25.478200 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.478703 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.479666 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.479863 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.479918 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.482202 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:25.482204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:25.482204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:25.482211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:25.483790 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.484467 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.484475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.484475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.484538 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.484949 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.484950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.484951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.485087 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:25.485147 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.485476 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.485563 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:06:25.485818 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.476313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:36829)
0926 13:06:25.476503 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.476542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.477125 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.477352 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.477356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.477924 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:25.478122 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.478127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.478784 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.478785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.479592 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.479594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.479935 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.479936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.480058 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.480548 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:25.480557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:25.482100 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:25.483884 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.483891 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.483893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.484211 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.484621 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:25.484622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:25.484623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:25.484797 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:25.485159 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:25.485166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:25.485347 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.485434 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.485558 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260926 13:06:25.486307 17136 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38481
I20260926 13:06:25.486415 17136 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:25.486613 17136 heartbeater.cc:511] Master 127.2.86.62:38481 requested a full tablet report, sending...
I20260926 13:06:25.487015 16678 ts_manager.cc:194] Registered new tserver with Master: ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663)
I20260926 13:06:25.487399 16678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:36829
I20260926 13:06:25.490972 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.484313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59926 (local address 127.2.86.62:38481)
0926 13:06:25.484425 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.484427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.484502 (+ 75us) server_negotiation.cc:407] Connection header received
0926 13:06:25.484556 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.484557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.484570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.484634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:25.485297 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.486003 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.487050 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.487303 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.487373 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.488591 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:25.488593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:25.488593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:25.488600 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:25.489825 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.490186 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.490187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.490188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.490248 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:25.490457 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:25.490458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:25.490458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:25.490568 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:25.490740 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.490776 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.490856 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260926 13:06:25.491657 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:06:25.494499 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.492003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.2.86.62:38481)
0926 13:06:25.492054 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.492056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.492230 (+ 174us) server_negotiation.cc:407] Connection header received
0926 13:06:25.492280 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.492280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.492306 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.492363 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:06:25.492439 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.492943 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.493777 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.494004 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.494063 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.494197 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:06:25.494286 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:06:25.494346 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.494383 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.494403 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 13:06:25.494786 16678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:06:25.496359 16678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:06:25.498530 16782 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2519970343410d84e735eeb12b7cb5 (DEFAULT_TABLE table=test-table [id=b7603002276f489397bb68e77d5143d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:25.498725 16782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2519970343410d84e735eeb12b7cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:25.500133 16758 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap starting.
I20260926 13:06:25.500530 16758 tablet_bootstrap.cc:654] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:25.501080 16758 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: No bootstrap required, opened a new log
I20260926 13:06:25.501123 16758 ts_tablet_manager.cc:1397] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.501928 16758 raft_consensus.cc:348] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.502039 16758 raft_consensus.cc:374] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:25.502095 16758 raft_consensus.cc:729] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89cc71e8e894a98a134dc16c126b6de, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.502146 16758 consensus_queue.cc:260] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.502265 16758 ts_tablet_manager.cc:1428] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 13:06:25.502772 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.498333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33663 (local address 127.0.0.1:58818)
0926 13:06:25.498372 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.498383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.498431 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.498963 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.498964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.498966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:25.499007 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.499008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.500059 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.500060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.501160 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.501163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.502564 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.502566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.502629 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.502633 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.502661 (+ 28us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.502702 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 13:06:25.503424 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.498518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.2.86.3:33663)
0926 13:06:25.498781 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.498786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.498798 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:06:25.498836 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.498838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.498856 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.498939 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:25.499120 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.499938 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.501435 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.502471 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.503013 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.503214 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.503243 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.503287 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260926 13:06:25.504309 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.498423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:33983 (local address 127.0.0.1:55560)
0926 13:06:25.500361 (+ 1938us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.500372 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.500433 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.500701 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.500702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.500704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:25.500739 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.500740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.501995 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.501996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.503063 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.503066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.504080 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.504081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.504146 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.504149 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.504204 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.504242 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1921,"thread_start_us":112,"threads_started":1}
I20260926 13:06:25.504334 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.498520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.2.86.2:33983)
0926 13:06:25.499079 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.499083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.500432 (+ 1349us) server_negotiation.cc:407] Connection header received
0926 13:06:25.500511 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.500514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.500530 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.500627 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:25.500805 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.501883 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.503281 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.503980 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.504064 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.504116 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.504220 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.504258 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":110,"threads_started":1}
I20260926 13:06:25.504356 17070 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2519970343410d84e735eeb12b7cb5 (DEFAULT_TABLE table=test-table [id=b7603002276f489397bb68e77d5143d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:25.504804 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2519970343410d84e735eeb12b7cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:25.505522 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2519970343410d84e735eeb12b7cb5 (DEFAULT_TABLE table=test-table [id=b7603002276f489397bb68e77d5143d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:25.505824 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2519970343410d84e735eeb12b7cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:25.508512 17146 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Bootstrap starting.
I20260926 13:06:25.509402 17146 tablet_bootstrap.cc:654] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:25.509608 17147 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Bootstrap starting.
I20260926 13:06:25.510232 17146 log.cc:824] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.510386 17147 tablet_bootstrap.cc:654] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:25.511003 17147 log.cc:824] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.511269 17146 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: No bootstrap required, opened a new log
I20260926 13:06:25.511350 17146 ts_tablet_manager.cc:1397] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:06:25.511893 17147 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: No bootstrap required, opened a new log
I20260926 13:06:25.511973 17147 ts_tablet_manager.cc:1397] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:25.512849 17146 raft_consensus.cc:348] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.513108 17146 raft_consensus.cc:374] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:25.513147 17146 raft_consensus.cc:729] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173852ee80a0430289e2cddfee8b1428, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.513262 17146 consensus_queue.cc:260] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.513576 17146 ts_tablet_manager.cc:1428] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:06:25.513576 17004 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
I20260926 13:06:25.513727 17147 raft_consensus.cc:348] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.513918 17147 raft_consensus.cc:374] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:25.513954 17147 raft_consensus.cc:729] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee14d479544749cc84f206d7461c4833, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.514091 17147 consensus_queue.cc:260] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.514407 17136 heartbeater.cc:503] Master 127.2.86.62:38481 was elected leader, sending a full tablet report...
I20260926 13:06:25.514392 17147 ts_tablet_manager.cc:1428] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260926 13:06:25.599858 17005 tablet.cc:2369] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:06:25.626592 17151 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:25.626731 17151 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.627151 17151 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983), c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747)
I20260926 13:06:25.631285 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.627472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:50185 (local address 127.2.86.2:33983)
0926 13:06:25.627534 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.627536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.627669 (+ 133us) server_negotiation.cc:407] Connection header received
0926 13:06:25.627698 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.627700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.627713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.627764 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:25.627946 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.628692 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630296 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.630877 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630956 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.630968 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.631167 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.631210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:06:25.631285 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.627563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:36019 (local address 127.2.86.1:41747)
0926 13:06:25.627612 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.627614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.627621 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:06:25.627658 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.627659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.627674 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.627734 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:25.627944 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.628696 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630299 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.630879 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630956 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.630968 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.631167 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.631210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:06:25.631302 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.627367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41747 (local address 127.2.86.3:36019)
0926 13:06:25.627432 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.627449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.627515 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.627771 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.627772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.627774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:25.627867 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.627869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.628786 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.628787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.630198 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.630199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.630972 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.631080 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.631082 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.631151 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.631207 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":132}
I20260926 13:06:25.631319 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.627367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:33983 (local address 127.2.86.3:50185)
0926 13:06:25.627586 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.627596 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.627661 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:25.627777 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:25.627777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:25.627779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:25.627877 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.627878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.628786 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.628787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.630195 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:25.630199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.630974 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.630975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:25.631076 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.631081 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:06:25.631151 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:06:25.631207 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":124,"thread_start_us":121,"threads_started":1}
I20260926 13:06:25.631713 16820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2519970343410d84e735eeb12b7cb5" candidate_uuid: "ee14d479544749cc84f206d7461c4833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c89cc71e8e894a98a134dc16c126b6de" is_pre_election: true
I20260926 13:06:25.631920 16820 raft_consensus.cc:2454] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee14d479544749cc84f206d7461c4833 in term 0.
I20260926 13:06:25.631973 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2519970343410d84e735eeb12b7cb5" candidate_uuid: "ee14d479544749cc84f206d7461c4833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173852ee80a0430289e2cddfee8b1428" is_pre_election: true
I20260926 13:06:25.632128 16958 raft_consensus.cc:2454] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee14d479544749cc84f206d7461c4833 in term 0.
I20260926 13:06:25.632153 17025 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de, ee14d479544749cc84f206d7461c4833; no voters:
I20260926 13:06:25.632362 17151 raft_consensus.cc:2781] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:06:25.632429 17151 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:06:25.632474 17151 raft_consensus.cc:3037] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:25.633330 17151 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.633481 17151 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [CANDIDATE]: Term 1 election: Requested vote from peers 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983), c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747)
I20260926 13:06:25.633633 16820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2519970343410d84e735eeb12b7cb5" candidate_uuid: "ee14d479544749cc84f206d7461c4833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c89cc71e8e894a98a134dc16c126b6de"
I20260926 13:06:25.633662 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2519970343410d84e735eeb12b7cb5" candidate_uuid: "ee14d479544749cc84f206d7461c4833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173852ee80a0430289e2cddfee8b1428"
I20260926 13:06:25.633741 16958 raft_consensus.cc:3037] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:25.633744 16820 raft_consensus.cc:3037] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:25.634444 16958 raft_consensus.cc:2454] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee14d479544749cc84f206d7461c4833 in term 1.
I20260926 13:06:25.634562 16820 raft_consensus.cc:2454] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee14d479544749cc84f206d7461c4833 in term 1.
I20260926 13:06:25.634632 17024 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833; no voters:
I20260926 13:06:25.634732 17151 raft_consensus.cc:2781] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:25.634867 17151 raft_consensus.cc:686] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 1 LEADER]: Becoming Leader. State: Replica: ee14d479544749cc84f206d7461c4833, State: Running, Role: LEADER
I20260926 13:06:25.634960 17151 consensus_queue.cc:237] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.635790 16678 catalog_manager.cc:5495] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee14d479544749cc84f206d7461c4833 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "ee14d479544749cc84f206d7461c4833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } health_report { overall_health: UNKNOWN } } }
I20260926 13:06:25.652386 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.649975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.2.86.3:33663)
0926 13:06:25.650040 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:25.650042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:25.650052 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:06:25.650115 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:25.650116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:25.650143 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:25.650217 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:06:25.650343 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.650827 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.651558 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:25.651757 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:25.651840 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:25.652024 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:06:25.652134 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:06:25.652228 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:25.652248 (+ 20us) server_negotiation.cc:299] Negotiation successful
0926 13:06:25.652269 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:06:25.654533 16958 raft_consensus.cc:1261] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Refusing update from remote peer ee14d479544749cc84f206d7461c4833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:25.654603 16820 raft_consensus.cc:1261] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Refusing update from remote peer ee14d479544749cc84f206d7461c4833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:25.654938 17151 consensus_queue.cc:1035] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:25.655087 17153 consensus_queue.cc:1035] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:25.657320 17157 mvcc.cc:205] Tried to move back new op lower bound from 7333593029237579776 to 7333593029161549824. Current Snapshot: MvccSnapshot[applied={T|T < 7333593029237579776}]
I20260926 13:06:25.658042 17159 mvcc.cc:205] Tried to move back new op lower bound from 7333593029237579776 to 7333593029161549824. Current Snapshot: MvccSnapshot[applied={T|T < 7333593029237579776}]
I20260926 13:06:25.659803 16677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:25.659897 16677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:25.660614 16677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Sending DeleteTablet for 3 replicas of tablet 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:25.660967 16938 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2519970343410d84e735eeb12b7cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:25 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:55560
I20260926 13:06:25.660993 17070 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2519970343410d84e735eeb12b7cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:25 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:58818
I20260926 13:06:25.661150 16782 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2519970343410d84e735eeb12b7cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:25 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:47056
I20260926 13:06:25.661175 17167 tablet_replica.cc:330] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: stopping tablet replica
I20260926 13:06:25.661237 16875 tablet_replica.cc:330] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: stopping tablet replica
I20260926 13:06:25.661281 16875 raft_consensus.cc:2229] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:25.661268 17167 raft_consensus.cc:2229] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:25.661341 16875 raft_consensus.cc:2258] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:25.661396 17167 raft_consensus.cc:2258] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:25.661473 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16716
I20260926 13:06:25.662789 17166 tablet_replica.cc:330] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: stopping tablet replica
I20260926 13:06:25.662912 17166 raft_consensus.cc:2229] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:06:25.663018 17166 raft_consensus.cc:2258] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:25.663010 17167 ts_tablet_manager.cc:1905] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:25.665685 17167 ts_tablet_manager.cc:1918] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:25.665773 17167 log.cc:1196] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:25.666091 17167 ts_tablet_manager.cc:1939] T 1c2519970343410d84e735eeb12b7cb5 P 173852ee80a0430289e2cddfee8b1428: Deleting consensus metadata
I20260926 13:06:25.666524 16663 catalog_manager.cc:4841] TS 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): tablet 1c2519970343410d84e735eeb12b7cb5 (table test-table [id=b7603002276f489397bb68e77d5143d7]) successfully deleted
I20260926 13:06:25.667054 17166 ts_tablet_manager.cc:1905] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:06:25.667909 16664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:41747 (error 108)
I20260926 13:06:25.668447 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16870
I20260926 13:06:25.669041 17166 ts_tablet_manager.cc:1918] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:25.669112 17166 log.cc:1196] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:25.669378 17166 ts_tablet_manager.cc:1939] T 1c2519970343410d84e735eeb12b7cb5 P ee14d479544749cc84f206d7461c4833: Deleting consensus metadata
I20260926 13:06:25.669732 16663 catalog_manager.cc:4841] TS ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): tablet 1c2519970343410d84e735eeb12b7cb5 (table test-table [id=b7603002276f489397bb68e77d5143d7]) successfully deleted
I20260926 13:06:25.673360 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:25.673215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:41747 (local address 127.0.0.1:47064)
0926 13:06:25.673276 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.673326 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:06:25.673528 16664 catalog_manager.cc:4584] TS c89cc71e8e894a98a134dc16c126b6de (127.2.86.1:41747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c2519970343410d84e735eeb12b7cb5: Network error: Client connection negotiation failed: client connection to 127.2.86.1:41747: connect: Connection refused (error 111)
I20260926 13:06:25.675940 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17007
I20260926 13:06:25.681281 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 16645
I20260926 13:06:25.687083 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--webserver_interface=127.2.86.62
--webserver_port=35293
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:38481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:25.770088 17169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.770314 17169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.770337 17169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.771546 17169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:25.771601 17169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.771617 17169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.771631 17169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:25.771642 17169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:25.773249 17169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:38481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.773517 17169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.773764 17169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:25.778950 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.778996 17169 server_base.cc:1034] running on GCE node
W20260926 13:06:25.778962 17174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.778959 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.779477 17169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.779814 17169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.780951 17169 hybrid_clock.cc:648] HybridClock initialized: now 1790427985780940 us; error 28 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.783597 17169 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:25.784142 17169 webserver.cc:460] Webserver started at http://127.2.86.62:35293/ using document root <none> and password file <none>
I20260926 13:06:25.784338 17169 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.784380 17169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.785816 17169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.786513 17184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.786734 17169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.786849 17169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "4a4e80cb46bd86baf091e2bd1d30267c"
server_key_iv: "71df5624e50165d17ec1613b78f00688"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.787209 17169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.800312 17169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.802798 17169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.802968 17169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.807078 17169 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:38481
I20260926 13:06:25.807128 17236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:38481 every 8 connection(s)
I20260926 13:06:25.807529 17169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:25.809263 17237 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:06:25.809983 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap starting.
I20260926 13:06:25.813004 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17169
I20260926 13:06:25.813315 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:41747
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=46859
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:06:25.819986 17237 log.cc:824] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.823604 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:06:25.824000 17237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2: Bootstrap complete.
I20260926 13:06:25.827418 17237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.827735 17237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.827894 17237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.827967 17237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:25.828006 17237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:25.828045 17237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:06:25.829429 17237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.829543 17237 leader_election.cc:302] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7709ae46f6fc4127ad3a0a5ee17eafd2; no voters:
I20260926 13:06:25.829712 17237 leader_election.cc:288] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 13:06:25.829802 17241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 13:06:25.829963 17237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:25.829963 17241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [term 3 LEADER]: Becoming Leader. State: Replica: 7709ae46f6fc4127ad3a0a5ee17eafd2, State: Running, Role: LEADER
I20260926 13:06:25.830292 17241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } }
I20260926 13:06:25.830641 17243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7709ae46f6fc4127ad3a0a5ee17eafd2. Latest consensus state: current_term: 3 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:25.830637 17242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7709ae46f6fc4127ad3a0a5ee17eafd2" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38481 } } }
I20260926 13:06:25.830755 17243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:25.830761 17242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7709ae46f6fc4127ad3a0a5ee17eafd2 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:25.830943 17250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:25.832336 17250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:25.832454 17250 catalog_manager.cc:1259] Loaded cluster ID: 2a878b40e3394eb5855a96f2f03cbd58
I20260926 13:06:25.832497 17250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 13:06:25.833009 17250 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff0cb98d9ad google::LogMessage::Fail() at ??:0
@ 0x7ff0cb99088a google::LogMessage::SendToLog() at ??:0
@ 0x7ff0cb98d4de google::LogMessage::Flush() at ??:0
@ 0x7ff0cb98ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff0d38ae0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff0d38b78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff0cd85b0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff0cd850954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff0ce6b46db start_thread at ??:0
@ 0x7ff0ca48b61f clone at ??:0
W20260926 13:06:25.930984 17239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:25.931418 17239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:25.931452 17239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:25.932849 17239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:25.932981 17239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:25.933017 17239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:25.934624 17239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:41747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=46859
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:25.934891 17239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:25.935184 17239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:06:25.942116 17239 server_base.cc:1034] running on GCE node
W20260926 13:06:25.942130 17274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.942065 17273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:25.942049 17276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:25.942512 17239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:25.942798 17239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:25.943964 17239 hybrid_clock.cc:648] HybridClock initialized: now 1790427985943934 us; error 61 us; skew 500 ppm
Sep 26 13:06:25 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:25.946818 17239 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:25.947402 17239 webserver.cc:460] Webserver started at http://127.2.86.1:46859/ using document root <none> and password file <none>
I20260926 13:06:25.947798 17239 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:25.947888 17239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:25.950002 17239 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:25.951023 17283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:25.951257 17239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:25.951388 17239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c89cc71e8e894a98a134dc16c126b6de"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "a48b1213ac5f1c5cc1e8a65a7e3dd0bd"
server_key_iv: "ce34395acbeda60531932549281fe3b9"
server_key_version: "encryptionkey@0"
I20260926 13:06:25.951977 17239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:25.966276 17239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:25.969365 17239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:25.969573 17239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:25.969933 17239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:25.970575 17290 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:06:25.972121 17239 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:06:25.972199 17239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 13:06:25.972239 17239 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:06:25.973330 17239 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:06:25.973382 17239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:25.973467 17290 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap starting.
I20260926 13:06:25.981778 17239 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:41747
I20260926 13:06:25.981848 17397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:41747 every 8 connection(s)
I20260926 13:06:25.982333 17239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:06:25.984367 17290 log.cc:824] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:25.986626 17290 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:06:25.987159 17290 tablet_bootstrap.cc:492] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Bootstrap complete.
I20260926 13:06:25.988184 17290 ts_tablet_manager.cc:1397] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260926 13:06:25.989383 17290 raft_consensus.cc:348] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.989648 17290 raft_consensus.cc:729] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89cc71e8e894a98a134dc16c126b6de, State: Initialized, Role: FOLLOWER
I20260926 13:06:25.989825 17290 consensus_queue.cc:260] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:25.990140 17290 ts_tablet_manager.cc:1428] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260926 13:06:25.990866 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17239
I20260926 13:06:25.991289 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:33983
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=46185
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:26.075922 17400 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:06:25.983239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:54553)
0926 13:06:25.983484 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:25.983547 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:25.984239 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:26.075682 (+ 91443us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":114,"threads_started":1}
W20260926 13:06:26.076216 17398 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:38481 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:38481: Client connection negotiation failed: client connection to 127.2.86.62:38481: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:06:26.076664 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.076544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:45001)
0926 13:06:26.076596 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.076643 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 13:06:26.077062 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.076961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:44121)
0926 13:06:26.077007 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.077038 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 13:06:26.077188 17398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:06:26.078244 17403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:26.078446 17403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:26.078474 17403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:26.079732 17403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:26.079777 17403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:26.079805 17403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:26.081409 17403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:33983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=46185
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:26.081677 17403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:26.081952 17403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:26.087255 17411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:26.087255 17409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:26.087282 17408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:26.087601 17403 server_base.cc:1034] running on GCE node
I20260926 13:06:26.087769 17403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:26.087980 17403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:26.089148 17403 hybrid_clock.cc:648] HybridClock initialized: now 1790427986089138 us; error 30 us; skew 500 ppm
Sep 26 13:06:26 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427986, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:26.091674 17403 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:26.092154 17403 webserver.cc:460] Webserver started at http://127.2.86.2:46185/ using document root <none> and password file <none>
I20260926 13:06:26.092403 17403 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:26.092471 17403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:26.093972 17403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:26.094676 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.094913 17403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:26.095019 17403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "173852ee80a0430289e2cddfee8b1428"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "07b7072a0843b3141825a58fb877089f"
server_key_iv: "6c74abe988233afa9feb8cafd5e8bb8f"
server_key_version: "encryptionkey@0"
I20260926 13:06:26.095408 17403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:26.114507 17403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:26.116669 17403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:26.116842 17403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:26.117187 17403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:26.117515 17403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:26.117565 17403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.117606 17403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:26.117638 17403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.123787 17403 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:33983
I20260926 13:06:26.123823 17531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:33983 every 8 connection(s)
I20260926 13:06:26.124285 17403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:06:26.125566 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.125062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:34123)
0926 13:06:26.125331 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.125408 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":79,"threads_started":1}
W20260926 13:06:26.126039 17532 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:38481 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:38481: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
I20260926 13:06:26.126343 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.126251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:35273)
0926 13:06:26.126295 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.126324 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 13:06:26.126611 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.126559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:54159)
0926 13:06:26.126580 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.126599 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260926 13:06:26.126708 17532 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:06:26.127346 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17403
I20260926 13:06:26.127624 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:33663
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:38481
--webserver_port=39295
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:26.210713 17535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:26.210947 17535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:26.210971 17535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:26.212283 17535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:26.212327 17535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:26.212356 17535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:26.213909 17535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=39295
--tserver_master_addrs=127.2.86.62:38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:26.214172 17535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:26.214417 17535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:26.219594 17541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:26.219697 17540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:26.219739 17535 server_base.cc:1034] running on GCE node
W20260926 13:06:26.219740 17543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:26.220150 17535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:26.220418 17535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:26.221585 17535 hybrid_clock.cc:648] HybridClock initialized: now 1790427986221579 us; error 34 us; skew 500 ppm
Sep 26 13:06:26 dist-test-slave-j4lq krb5kdc[16097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427986, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:26.224431 17535 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:26.224972 17535 webserver.cc:460] Webserver started at http://127.2.86.3:39295/ using document root <none> and password file <none>
I20260926 13:06:26.225200 17535 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:26.225263 17535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:26.226608 17535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:06:26.227268 17550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.227432 17535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:06:26.227543 17535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "ee14d479544749cc84f206d7461c4833"
format_stamp: "Formatted at 2026-09-26 13:06:24 on dist-test-slave-j4lq"
server_key: "3ea768fdece359773141398ece91c9a1"
server_key_iv: "0ce318b7a8636ca88330c171c597c0b1"
server_key_version: "encryptionkey@0"
I20260926 13:06:26.227905 17535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:26.238536 17535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:26.240483 17535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:26.240635 17535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:26.240852 17535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:26.241214 17535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:26.241245 17535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.241268 17535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:26.241281 17535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:26.246866 17535 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:33663
I20260926 13:06:26.246937 17663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:33663 every 8 connection(s)
I20260926 13:06:26.247267 17535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:06:26.248311 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.247919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:53387)
0926 13:06:26.248077 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.248171 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260926 13:06:26.248729 17664 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:38481 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:38481: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
I20260926 13:06:26.249049 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.248962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:51843)
0926 13:06:26.249005 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.249032 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:06:26.249292 17666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:26.249241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:34877)
0926 13:06:26.249261 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:26.249280 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260926 13:06:26.249454 17664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:06:26.252609 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17535
I20260926 13:06:27.077970 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.077631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:59863)
0926 13:06:27.077816 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:27.077863 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260926 13:06:27.127460 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.127132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:49213)
0926 13:06:27.127333 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:27.127393 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260926 13:06:27.250108 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.249829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:48693)
0926 13:06:27.250013 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:27.250054 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
I20260926 13:06:27.411824 17671 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:27.411962 17671 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:27.412442 17671 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
Sep 26 13:06:27 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.2@KRBTEST.COM
Sep 26 13:06:27 dist-test-slave-j4lq krb5kdc[16097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790427985, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.3@KRBTEST.COM
I20260926 13:06:27.424194 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.412919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:51557 (local address 127.2.86.2:33983)
0926 13:06:27.413037 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:27.413048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:27.413093 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:06:27.413540 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:27.413542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:27.413586 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:27.413766 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:27.414501 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.415492 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.416466 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.416992 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.417123 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:27.419942 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:27.419946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:27.419947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:27.419959 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:27.422217 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422753 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:27.422756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:27.422757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:27.422870 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:27.423158 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:27.423159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:27.423159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:27.423457 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:27.423530 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:27.423885 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 13:06:27.424040 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":54,"threads_started":1}
I20260926 13:06:27.424194 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.412734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:42105 (local address 127.2.86.3:33663)
0926 13:06:27.412933 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:27.412946 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:27.412986 (+ 40us) server_negotiation.cc:407] Connection header received
0926 13:06:27.413490 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:27.413492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:27.413537 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:27.413664 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:27.414503 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.415384 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.416492 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.416977 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.417083 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:27.419965 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:27.419969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:27.419969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:27.419979 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:27.421822 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422357 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:27.422360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:27.422361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:27.422494 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422995 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:27.422996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:27.422996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:27.423391 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:27.423471 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:27.423903 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 13:06:27.424040 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260926 13:06:27.424261 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.412794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:33983 (local address 127.2.86.1:51557)
0926 13:06:27.412947 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:27.412958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:27.413010 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:27.413762 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:27.413764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:27.414173 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:27.414372 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:27.414378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.415596 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.415597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:27.416378 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:27.416380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.417093 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.417094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:27.417202 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:27.417851 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:27.417860 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:27.419504 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:27.422340 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:27.422342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:27.422629 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:27.422986 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:27.422987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:27.423087 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:27.423523 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:27.423524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:27.423679 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 13:06:27.423814 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:06:27.424108 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":47,"thread_start_us":111,"threads_started":1}
I20260926 13:06:27.424985 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:27.412635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:33663 (local address 127.2.86.1:42105)
0926 13:06:27.412686 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:27.412701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:27.412775 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:27.413742 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:27.413746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:27.414173 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:27.414372 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:27.414378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.415497 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.415499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:27.416379 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:27.416382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:27.417071 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:27.417073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:27.417201 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:27.417917 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:27.417922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:27.419834 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:27.421916 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:27.421926 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:27.421929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:27.422271 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:27.422612 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:27.422613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:27.422614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:27.422753 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:27.423466 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:27.423473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:27.423679 (+ 206us) client_negotiation.cc:769] Sending connection context
0926 13:06:27.423814 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:06:27.424171 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":272}
W20260926 13:06:27.425660 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
W20260926 13:06:27.426010 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:27.426059 17285 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de; no voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833
I20260926 13:06:27.426196 17671 raft_consensus.cc:2726] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:06:28.078722 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:28.078487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:39275)
0926 13:06:28.078640 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:28.078675 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260926 13:06:28.128239 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:28.128036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:35055)
0926 13:06:28.128157 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:28.128199 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260926 13:06:28.250877 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:28.250638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:34935)
0926 13:06:28.250778 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:28.250833 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260926 13:06:29.079466 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:29.079231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:46479)
0926 13:06:29.079373 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:29.079419 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260926 13:06:29.128893 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:29.128687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:49799)
0926 13:06:29.128801 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:29.128837 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260926 13:06:29.251667 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:29.251377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:32769)
0926 13:06:29.251567 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:29.251609 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260926 13:06:29.573894 17681 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:29.573990 17681 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:29.574218 17681 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
W20260926 13:06:29.574595 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
W20260926 13:06:29.574664 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:29.574681 17285 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de; no voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833
I20260926 13:06:29.574747 17681 raft_consensus.cc:2726] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:06:30.080237 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:30.080001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:55193)
0926 13:06:30.080139 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:30.080184 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260926 13:06:30.129526 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:30.129334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:44437)
0926 13:06:30.129445 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:30.129484 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260926 13:06:30.252490 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:30.252253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:41011)
0926 13:06:30.252411 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:30.252445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260926 13:06:31.081115 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:31.080777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:53823)
0926 13:06:31.080978 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:31.081046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":64,"threads_started":1}
I20260926 13:06:31.130239 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:31.129984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:56329)
0926 13:06:31.130118 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:31.130162 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260926 13:06:31.253257 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:31.253030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:57989)
0926 13:06:31.253171 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:31.253213 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260926 13:06:32.081910 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:32.081651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:60971)
0926 13:06:32.081812 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:32.081864 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260926 13:06:32.130872 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:32.130677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:46879)
0926 13:06:32.130791 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:32.130827 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260926 13:06:32.253958 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:32.253734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:47653)
0926 13:06:32.253872 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:32.253914 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260926 13:06:32.390735 17691 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:32.390836 17691 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:32.391098 17691 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
W20260926 13:06:32.391587 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
W20260926 13:06:32.391659 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:32.391702 17285 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de; no voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833
I20260926 13:06:32.391801 17691 raft_consensus.cc:2726] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:06:33.082603 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:33.082389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:34491)
0926 13:06:33.082511 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:33.082549 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260926 13:06:33.131513 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:33.131313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:34847)
0926 13:06:33.131431 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:33.131472 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
I20260926 13:06:33.254657 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:33.254436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:54583)
0926 13:06:33.254571 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:33.254611 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260926 13:06:34.083487 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:34.083196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:53313)
0926 13:06:34.083370 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:34.083422 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260926 13:06:34.132294 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:34.132039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:57561)
0926 13:06:34.132187 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:34.132249 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260926 13:06:34.255473 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:34.255221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:36441)
0926 13:06:34.255369 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:34.255423 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260926 13:06:35.084337 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:35.084089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:33247)
0926 13:06:35.084231 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:35.084292 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260926 13:06:35.133092 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:35.132800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:49745)
0926 13:06:35.132993 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:35.133045 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260926 13:06:35.256209 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:35.255943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:45009)
0926 13:06:35.256123 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:35.256163 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260926 13:06:35.528513 17701 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:35.528616 17701 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:35.528839 17701 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
W20260926 13:06:35.529403 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
W20260926 13:06:35.529505 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:35.529546 17285 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de; no voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833
I20260926 13:06:35.529630 17701 raft_consensus.cc:2726] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:06:36.085311 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:36.085102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:49369)
0926 13:06:36.085225 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:36.085266 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260926 13:06:36.133833 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:36.133590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:53991)
0926 13:06:36.133738 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:36.133778 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260926 13:06:36.257014 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:36.256701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:58623)
0926 13:06:36.256849 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:36.256969 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260926 13:06:37.086024 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:37.085789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:35235)
0926 13:06:37.085938 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:37.085977 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260926 13:06:37.134552 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:37.134349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:41491)
0926 13:06:37.134468 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:37.134509 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260926 13:06:37.257920 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:37.257661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:34883)
0926 13:06:37.257824 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:37.257872 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260926 13:06:38.086810 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:38.086571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:54155)
0926 13:06:38.086719 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:38.086764 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260926 13:06:38.135308 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:38.135064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:59229)
0926 13:06:38.135221 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:38.135263 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260926 13:06:38.215917 17710 raft_consensus.cc:482] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:38.216061 17710 raft_consensus.cc:504] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee14d479544749cc84f206d7461c4833" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 33663 } } peers { permanent_uuid: "173852ee80a0430289e2cddfee8b1428" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 33983 } } peers { permanent_uuid: "c89cc71e8e894a98a134dc16c126b6de" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 41747 } }
I20260926 13:06:38.216259 17710 leader_election.cc:288] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663), 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983)
W20260926 13:06:38.216642 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee14d479544749cc84f206d7461c4833 (127.2.86.3:33663): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
W20260926 13:06:38.216722 17285 leader_election.cc:341] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173852ee80a0430289e2cddfee8b1428 (127.2.86.2:33983): Not found: Tablet not found: 1c2519970343410d84e735eeb12b7cb5
I20260926 13:06:38.216770 17285 leader_election.cc:302] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c89cc71e8e894a98a134dc16c126b6de; no voters: 173852ee80a0430289e2cddfee8b1428, ee14d479544749cc84f206d7461c4833
I20260926 13:06:38.216861 17710 raft_consensus.cc:2726] T 1c2519970343410d84e735eeb12b7cb5 P c89cc71e8e894a98a134dc16c126b6de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:06:38.258692 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:38.258448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:59733)
0926 13:06:38.258600 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:38.258644 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260926 13:06:39.087610 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:39.087370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:57765)
0926 13:06:39.087523 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:39.087559 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260926 13:06:39.136030 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:39.135795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:43655)
0926 13:06:39.135947 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:39.135986 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260926 13:06:39.259506 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:39.259259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:34103)
0926 13:06:39.259399 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:39.259444 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260926 13:06:40.088490 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:40.088232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:48337)
0926 13:06:40.088411 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:40.088442 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260926 13:06:40.136755 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:40.136498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:56455)
0926 13:06:40.136664 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:40.136708 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260926 13:06:40.260213 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:40.259995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:47699)
0926 13:06:40.260126 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:40.260167 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260926 13:06:41.089389 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:41.089120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.1:48145)
0926 13:06:41.089287 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:41.089345 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":69,"threads_started":1}
I20260926 13:06:41.137635 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:41.137381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.2:54183)
0926 13:06:41.137553 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:41.137592 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260926 13:06:41.261065 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:41.260802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38481 (local address 127.2.86.3:60533)
0926 13:06:41.260976 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:41.261019 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260926 13:06:41.263084 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17239
I20260926 13:06:41.267953 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17403
I20260926 13:06:41.272465 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17535
2026-09-26T13:06:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18410 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:06:41 dist-test-slave-j4lq krb5kdc[17723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:06:41 dist-test-slave-j4lq krb5kdc[17723](info): set up 2 sockets
Sep 26 13:06:41 dist-test-slave-j4lq krb5kdc[17723](info): commencing operation
krb5kdc: starting...
W20260926 13:06:42.321151 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:06:42.446045 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:42.529544 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:42.529845 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:42.529887 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:42.531175 17736 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:06:42.531252 17736 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:06:42.531275 17736 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:06:42.531296 17736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:06:42.531327 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:42.531358 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:42.531385 17736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:06:42.531411 17736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:06:42.533109 17736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:42.533419 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:42.533674 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:42.538894 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:42.538880 17744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:42.539057 17736 server_base.cc:1034] running on GCE node
W20260926 13:06:42.538884 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:42.539510 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:42.539860 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:42.541036 17736 hybrid_clock.cc:648] HybridClock initialized: now 1790428002541003 us; error 57 us; skew 500 ppm
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:42.543691 17736 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:06:42.544185 17736 webserver.cc:460] Webserver started at http://127.2.86.62:36277/ using document root <none> and password file <none>
I20260926 13:06:42.544376 17736 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:42.544420 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:42.544508 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:42.545589 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "15ecdf08525b43a2921e8e9c556fae83"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "2e47202369ca86f18ce74d7aad0166bb"
server_key_iv: "573d20e9b44a4485125846e26e15bb17"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.545904 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "15ecdf08525b43a2921e8e9c556fae83"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "2e47202369ca86f18ce74d7aad0166bb"
server_key_iv: "573d20e9b44a4485125846e26e15bb17"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.547210 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:06:42.547951 17751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.548151 17736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:42.548241 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "15ecdf08525b43a2921e8e9c556fae83"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "2e47202369ca86f18ce74d7aad0166bb"
server_key_iv: "573d20e9b44a4485125846e26e15bb17"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.548300 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:42.572539 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:42.575091 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:42.575213 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:42.579345 17736 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33413
I20260926 13:06:42.579373 17803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33413 every 8 connection(s)
I20260926 13:06:42.579769 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:06:42.580721 17804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:42.581436 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17736
I20260926 13:06:42.581518 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:06:42.581734 2392 external_mini_cluster.cc:1428] Setting key 046d0a0943e0acdba6cd6750872b4c91
I20260926 13:06:42.583890 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap starting.
I20260926 13:06:42.584633 17804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:42.584961 17804 log.cc:824] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:42.585661 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: No bootstrap required, opened a new log
I20260926 13:06:42.587812 17804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:06:42.587951 17804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:42.587973 17804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Initialized, Role: FOLLOWER
I20260926 13:06:42.588122 17804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:06:42.588201 17804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:06:42.588243 17804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:06:42.588295 17804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:42.588932 17804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:42.589048 17804 leader_election.cc:302] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ecdf08525b43a2921e8e9c556fae83; no voters:
I20260926 13:06:42.589250 17804 leader_election.cc:288] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:06:42.589305 17809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:42.589453 17809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 LEADER]: Becoming Leader. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Running, Role: LEADER
I20260926 13:06:42.589541 17804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:06:42.589608 17809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:06:42.590306 17809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ecdf08525b43a2921e8e9c556fae83. Latest consensus state: current_term: 1 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:06:42.590314 17811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:06:42.590409 17811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:42.590832 17809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:06:42.590904 17821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:06:42.591559 17821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:06:42.593390 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:42.582273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55350 (local address 127.2.86.62:33413)
0926 13:06:42.582470 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:42.582480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:42.582514 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:06:42.583104 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:42.583114 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:42.583184 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:42.583343 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:42.583931 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.585314 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.586151 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.586779 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.586900 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:42.589556 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:42.589561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:42.589561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:42.589587 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:42.591820 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.592179 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.592183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.592184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.592288 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.592504 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.592504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.592504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.592762 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:42.592984 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:42.593079 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:06:42.593218 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260926 13:06:42.593716 17821 catalog_manager.cc:1347] Generated new cluster ID: 24c071ec10ec4881a4e5afb536c55c09
I20260926 13:06:42.593765 17821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:06:42.607772 17821 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:06:42.608517 17821 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:06:42.619544 17821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Generated new TSK 0
I20260926 13:06:42.619788 17821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:06:42.650147 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:42.762369 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:42.762640 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:42.762689 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:42.764002 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:42.764073 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:42.764124 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:06:42.766130 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:42.766428 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:42.766686 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:42.771699 17835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:42.771703 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:42.771746 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:42.771967 17830 server_base.cc:1034] running on GCE node
I20260926 13:06:42.772171 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:42.772431 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:42.773607 17830 hybrid_clock.cc:648] HybridClock initialized: now 1790428002773594 us; error 44 us; skew 500 ppm
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:42.776132 17830 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:06:42.776598 17830 webserver.cc:460] Webserver started at http://127.2.86.1:35287/ using document root <none> and password file <none>
I20260926 13:06:42.776844 17830 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:42.776954 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:42.777110 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:42.778185 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "5e7458483e784255a4e37a60b8f31bad"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "4328da9b7e9e0c767b1fd54cd0d46c5b"
server_key_iv: "62a6f0deac9317a9ff8a5421b899b0db"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.778554 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "5e7458483e784255a4e37a60b8f31bad"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "4328da9b7e9e0c767b1fd54cd0d46c5b"
server_key_iv: "62a6f0deac9317a9ff8a5421b899b0db"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.779882 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:06:42.780596 17845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.780773 17830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:06:42.780865 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "5e7458483e784255a4e37a60b8f31bad"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "4328da9b7e9e0c767b1fd54cd0d46c5b"
server_key_iv: "62a6f0deac9317a9ff8a5421b899b0db"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.780978 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:42.790148 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:42.792575 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:42.792753 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:42.793054 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:42.793370 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:42.793424 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.793467 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:42.793499 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.799434 17830 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:33303
I20260926 13:06:42.799485 17958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:33303 every 8 connection(s)
I20260926 13:06:42.799817 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:42.806252 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17830
I20260926 13:06:42.806365 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:06:42.806577 2392 external_mini_cluster.cc:1428] Setting key 6902f0b154b4265c5135ff66fafe4671
I20260926 13:06:42.810734 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:42.800529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:42957 (local address 127.2.86.62:33413)
0926 13:06:42.800621 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:42.800624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:42.800805 (+ 181us) server_negotiation.cc:407] Connection header received
0926 13:06:42.801334 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:42.801335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:42.801344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:42.801397 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:42.802311 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.802823 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.803777 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.803968 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.804019 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:42.806449 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:42.806451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:42.806451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:42.806459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:42.809214 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.809730 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.809731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.809731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.809782 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.810065 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.810065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.810066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.810180 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:42.810229 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:42.810588 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 13:06:42.810665 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:06:42.811127 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:42.800515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:42957)
0926 13:06:42.800672 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:42.800713 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:42.801245 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:42.801452 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:42.801455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:42.802030 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:42.802231 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:42.802235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.802908 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.802909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:42.803703 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:42.803704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.804038 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.804039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:42.804135 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:42.804707 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:42.804715 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:42.806355 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:42.809310 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:42.809317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:42.809319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:42.809646 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:42.809850 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:42.809851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:42.809851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:42.809998 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:42.810243 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:42.810250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:42.810446 (+ 196us) client_negotiation.cc:769] Sending connection context
0926 13:06:42.810566 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 13:06:42.810770 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260926 13:06:42.811671 17959 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:06:42.811798 17959 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:42.811996 17959 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:06:42.812520 17768 ts_manager.cc:194] Registered new tserver with Master: 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303)
I20260926 13:06:42.813215 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:42957
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:06:42.832554 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:42.913728 17964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:42.913980 17964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:42.914019 17964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:42.915149 17964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:42.915218 17964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:42.915262 17964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:06:42.916702 17964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:42.917002 17964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:42.917250 17964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:42.922168 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:42.922235 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:42.922291 17964 server_base.cc:1034] running on GCE node
W20260926 13:06:42.922168 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:42.922533 17964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:42.922806 17964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:42.923978 17964 hybrid_clock.cc:648] HybridClock initialized: now 1790428002923961 us; error 32 us; skew 500 ppm
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:42.926558 17964 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:06:42.927054 17964 webserver.cc:460] Webserver started at http://127.2.86.2:38847/ using document root <none> and password file <none>
I20260926 13:06:42.927238 17964 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:42.927278 17964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:42.927374 17964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:42.928421 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "be5e2e5d7f29d3eca9a0b662f09ed9a5"
server_key_iv: "2ad7d30482f63ee212844490125b7cbc"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.928725 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "be5e2e5d7f29d3eca9a0b662f09ed9a5"
server_key_iv: "2ad7d30482f63ee212844490125b7cbc"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.930150 17964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:42.930971 17979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.931154 17964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:06:42.931262 17964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "be5e2e5d7f29d3eca9a0b662f09ed9a5"
server_key_iv: "2ad7d30482f63ee212844490125b7cbc"
server_key_version: "encryptionkey@0"
I20260926 13:06:42.931335 17964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:42.942621 17964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:42.945087 17964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:42.945263 17964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:42.945503 17964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:42.945823 17964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:42.945878 17964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.945921 17964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:42.945946 17964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:42.951887 17964 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:35499
I20260926 13:06:42.951967 18092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:35499 every 8 connection(s)
I20260926 13:06:42.952327 17964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:06:42.957412 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 17964
I20260926 13:06:42.957530 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:06:42.957753 2392 external_mini_cluster.cc:1428] Setting key 947404775503f9c6838a9c48dab4f38f
Sep 26 13:06:42 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:42.961928 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:42.953116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:34331 (local address 127.2.86.62:33413)
0926 13:06:42.953195 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:42.953197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:42.953390 (+ 193us) server_negotiation.cc:407] Connection header received
0926 13:06:42.953843 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:42.953844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:42.953854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:42.953904 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:42.954813 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.955305 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.956252 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.956427 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.956477 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:42.958947 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:42.958948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:42.958949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:42.958957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:42.960473 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.960999 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.961000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.961000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.961050 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:42.961327 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:42.961328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:42.961328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:42.961429 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:42.961479 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:42.961800 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 13:06:42.961862 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 13:06:42.962255 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:42.953092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:34331)
0926 13:06:42.953259 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:42.953297 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:42.953758 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:42.953966 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:42.953970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:42.954537 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:42.954736 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:42.954741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.955384 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.955385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:42.956178 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:42.956180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:42.956496 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:42.956497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:42.956591 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:42.957147 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:42.957155 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:42.958853 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:42.960555 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:42.960562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:42.960564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:42.960921 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:42.961120 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:42.961121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:42.961121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:42.961264 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:42.961491 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:42.961496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:42.961667 (+ 171us) client_negotiation.cc:769] Sending connection context
0926 13:06:42.961780 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 13:06:42.961955 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260926 13:06:42.962771 18093 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:06:42.962894 18093 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:42.963070 18093 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:06:42.963488 17768 ts_manager.cc:194] Registered new tserver with Master: eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
I20260926 13:06:42.963863 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:34331
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:06:42.983940 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:06:43.070598 18098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:06:43.070868 18098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:06:43.070911 18098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:06:43.072212 18098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:06:43.072286 18098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:06:43.072331 18098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:06:43.074461 18098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:06:43.074748 18098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:06:43.075047 18098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:06:43.081676 18106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:06:43.081677 18103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:43.081888 18098 server_base.cc:1034] running on GCE node
W20260926 13:06:43.081682 18104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:06:43.082230 18098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:06:43.082556 18098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:06:43.083747 18098 hybrid_clock.cc:648] HybridClock initialized: now 1790428003083733 us; error 33 us; skew 500 ppm
Sep 26 13:06:43 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428003, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:06:43.086582 18098 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:06:43.087111 18098 webserver.cc:460] Webserver started at http://127.2.86.3:39635/ using document root <none> and password file <none>
I20260926 13:06:43.087355 18098 fs_manager.cc:359] Metadata directory not provided
I20260926 13:06:43.087423 18098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:06:43.087559 18098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:06:43.088624 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "5e2ad162cd5f408d9a4175b058565226"
format_stamp: "Formatted at 2026-09-26 13:06:43 on dist-test-slave-j4lq"
server_key: "d3cabf1699f61266e9551368acd23b88"
server_key_iv: "dbb79374fc169a49236f265da10fc9ac"
server_key_version: "encryptionkey@0"
I20260926 13:06:43.089035 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "5e2ad162cd5f408d9a4175b058565226"
format_stamp: "Formatted at 2026-09-26 13:06:43 on dist-test-slave-j4lq"
server_key: "d3cabf1699f61266e9551368acd23b88"
server_key_iv: "dbb79374fc169a49236f265da10fc9ac"
server_key_version: "encryptionkey@0"
I20260926 13:06:43.090667 18098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:43.091634 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:43.091828 18098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:06:43.091928 18098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "5e2ad162cd5f408d9a4175b058565226"
format_stamp: "Formatted at 2026-09-26 13:06:43 on dist-test-slave-j4lq"
server_key: "d3cabf1699f61266e9551368acd23b88"
server_key_iv: "dbb79374fc169a49236f265da10fc9ac"
server_key_version: "encryptionkey@0"
I20260926 13:06:43.092041 18098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:06:43.113686 18098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:06:43.116340 18098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:06:43.116523 18098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:06:43.116787 18098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:06:43.117151 18098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:06:43.117211 18098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:43.117254 18098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:06:43.117280 18098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:06:43.124064 18098 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38343
I20260926 13:06:43.124107 18226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38343 every 8 connection(s)
I20260926 13:06:43.124457 18098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:06:43.129024 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18098
I20260926 13:06:43.129143 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:06:43.129362 2392 external_mini_cluster.cc:1428] Setting key f9e0953cb3dc384cc37f394286f811a2
Sep 26 13:06:43 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428003, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:06:43.134276 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.125200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:41519 (local address 127.2.86.62:33413)
0926 13:06:43.125256 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.125258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.125433 (+ 175us) server_negotiation.cc:407] Connection header received
0926 13:06:43.125927 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.125928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.125937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.125998 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:43.126901 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.127419 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.128382 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.128562 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.128613 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.131232 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:43.131233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:43.131233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:43.131240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:43.132638 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:43.133149 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:43.133150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:43.133150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:43.133204 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:43.133525 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:43.133526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:43.133526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:43.133640 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:43.133691 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.134124 (+ 433us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.134211 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:06:43.134495 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.125151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:41519)
0926 13:06:43.125316 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.125360 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.125845 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.126072 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.126075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.126636 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:06:43.126825 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.126830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.127502 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.127503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.128308 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.128310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.128633 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.128634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.128730 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.129557 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:06:43.129585 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:06:43.131130 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:06:43.132735 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:43.132741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:43.132743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:43.133076 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:43.133291 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:06:43.133292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:06:43.133293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:06:43.133401 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:06:43.133745 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:06:43.133751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:06:43.133935 (+ 184us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.134075 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.134231 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260926 13:06:43.135097 18227 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:06:43.135250 18227 heartbeater.cc:464] Registering TS with master...
I20260926 13:06:43.135397 18227 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:06:43.135834 17768 ts_manager.cc:194] Registered new tserver with Master: 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343)
I20260926 13:06:43.136264 17768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:41519
I20260926 13:06:43.141230 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:06:43.148546 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.142638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55362 (local address 127.2.86.62:33413)
0926 13:06:43.142677 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.142679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.142789 (+ 110us) server_negotiation.cc:407] Connection header received
0926 13:06:43.142842 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.142843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.142852 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.142909 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:06:43.143433 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.143899 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.144650 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.144836 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.145010 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.146129 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:06:43.146130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:06:43.146130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:06:43.146137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:06:43.147534 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:43.147840 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:43.147840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:43.147841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:43.147890 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:06:43.148072 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:06:43.148073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:06:43.148074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:06:43.148230 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:06:43.148324 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.148408 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.148480 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:06:43.150377 17768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:06:43.152065 17768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:06:43.158979 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.154982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38343 (local address 127.0.0.1:47264)
0926 13:06:43.155398 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.155407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.155444 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.155703 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.155704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.155706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:43.155765 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.155767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156620 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.156621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.157721 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.157723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158646 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.158710 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158715 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.158756 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.158815 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":34,"thread_start_us":69,"threads_started":1}
I20260926 13:06:43.158995 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.154945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.2.86.1:33303)
0926 13:06:43.155195 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.155199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.155321 (+ 122us) server_negotiation.cc:407] Connection header received
0926 13:06:43.155379 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.155381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.155397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.155481 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:43.155680 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156732 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158053 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158519 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158611 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158674 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.158866 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.158908 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
I20260926 13:06:43.159134 18243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.155097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.2.86.3:38343)
0926 13:06:43.155559 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.155563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.155570 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:06:43.155603 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.155605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.155616 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.155704 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:43.155823 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156517 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.157810 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158539 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158902 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158963 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.158990 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.159035 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":84,"threads_started":1}
I20260926 13:06:43.159157 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.154718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.2.86.2:35499)
0926 13:06:43.154944 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.154950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.154961 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:06:43.155007 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.155010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.155028 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.155116 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:43.155315 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156429 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158015 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158697 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158762 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158816 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.158976 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.159028 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260926 13:06:43.159509 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.154810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33303 (local address 127.0.0.1:46386)
0926 13:06:43.155244 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.155255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.155318 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.155562 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.155563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.155566 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:43.155607 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.155609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156831 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.156832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.157961 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.157963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158613 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.158694 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158707 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.158776 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.158821 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":77,"threads_started":1}
I20260926 13:06:43.159653 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.154578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.0.0.1:45700)
0926 13:06:43.154735 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.154769 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.154854 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.155118 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.155121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.155123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:43.155197 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.155199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.156553 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.156555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.157918 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.157921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.158772 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.158773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.158825 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.158828 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.158875 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.158910 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260926 13:06:43.160251 17893 tablet_service.cc:1467] Processing CreateTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (DEFAULT_TABLE table=test-table [id=b826609e73884d5aae23362f944e0329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:43.160249 18161 tablet_service.cc:1467] Processing CreateTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (DEFAULT_TABLE table=test-table [id=b826609e73884d5aae23362f944e0329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:43.160595 17893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:43.160606 18161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:43.161577 18027 tablet_service.cc:1467] Processing CreateTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (DEFAULT_TABLE table=test-table [id=b826609e73884d5aae23362f944e0329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:06:43.161899 18027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:06:43.163338 18244 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Bootstrap starting.
I20260926 13:06:43.164054 18244 tablet_bootstrap.cc:654] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:43.164057 18245 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap starting.
I20260926 13:06:43.164677 18244 log.cc:824] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:43.164743 18245 tablet_bootstrap.cc:654] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:43.165002 18246 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Bootstrap starting.
I20260926 13:06:43.165306 18245 log.cc:824] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:43.165710 18246 tablet_bootstrap.cc:654] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Neither blocks nor log segments found. Creating new log.
I20260926 13:06:43.165977 18244 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: No bootstrap required, opened a new log
I20260926 13:06:43.166028 18245 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: No bootstrap required, opened a new log
I20260926 13:06:43.166054 18244 ts_tablet_manager.cc:1397] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:06:43.166076 18245 ts_tablet_manager.cc:1397] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:06:43.166236 18246 log.cc:824] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:06:43.166848 18246 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: No bootstrap required, opened a new log
I20260926 13:06:43.166893 18246 ts_tablet_manager.cc:1397] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:06:43.168023 18245 raft_consensus.cc:348] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168023 18244 raft_consensus.cc:348] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168205 18245 raft_consensus.cc:374] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:43.168205 18244 raft_consensus.cc:374] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:43.168149 18246 raft_consensus.cc:348] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168242 18245 raft_consensus.cc:729] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7458483e784255a4e37a60b8f31bad, State: Initialized, Role: FOLLOWER
I20260926 13:06:43.168242 18244 raft_consensus.cc:729] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2ad162cd5f408d9a4175b058565226, State: Initialized, Role: FOLLOWER
I20260926 13:06:43.168267 18246 raft_consensus.cc:374] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:06:43.168327 18246 raft_consensus.cc:729] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda011b97b9241c3864a10bd7ebd0a5d, State: Initialized, Role: FOLLOWER
I20260926 13:06:43.168370 18244 consensus_queue.cc:260] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168378 18245 consensus_queue.cc:260] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168500 18246 consensus_queue.cc:260] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.168649 18245 ts_tablet_manager.cc:1428] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 13:06:43.168649 18244 ts_tablet_manager.cc:1428] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260926 13:06:43.168766 17959 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
I20260926 13:06:43.168774 18246 ts_tablet_manager.cc:1428] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:06:43.168921 18093 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
I20260926 13:06:43.169128 18227 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
W20260926 13:06:43.202948 18094 tablet.cc:2369] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:06:43.300570 17960 tablet.cc:2369] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:06:43.361819 18251 raft_consensus.cc:482] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:06:43.361945 18251 raft_consensus.cc:504] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.362375 18251 leader_election.cc:288] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499), 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303)
I20260926 13:06:43.365258 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.362670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:53979 (local address 127.2.86.2:35499)
0926 13:06:43.362744 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.362746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.362758 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:06:43.362791 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.362792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.362803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.362888 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:43.363075 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.363543 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.364463 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.364841 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.364976 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.364986 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.365152 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.365193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:06:43.365258 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.362589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.2.86.3:53979)
0926 13:06:43.362657 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.362670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.362717 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.362894 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.362895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.362897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:43.362974 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.362984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.363645 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.363646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.364362 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.364364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.364985 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.364986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.365061 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.365065 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.365142 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.365187 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 13:06:43.365496 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.362727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33303 (local address 127.2.86.3:48891)
0926 13:06:43.362932 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.362939 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 13:06:43.363003 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:06:43.363155 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:06:43.363156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:06:43.363158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:06:43.363233 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.363234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.363849 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.363849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.364585 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:06:43.364586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.365199 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.365200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:06:43.365279 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.365282 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:06:43.365349 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:06:43.365395 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":127,"threads_started":1}
I20260926 13:06:43.365613 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.362803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:48891 (local address 127.2.86.1:33303)
0926 13:06:43.362873 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.362875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.362991 (+ 116us) server_negotiation.cc:407] Connection header received
0926 13:06:43.363056 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.363057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.363067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.363140 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:06:43.363309 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.363766 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.364677 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.365114 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.365196 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.365205 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.365474 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.365524 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:06:43.365643 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65dc6a0c0771435ba96ac9e2d19a2ff2" candidate_uuid: "5e2ad162cd5f408d9a4175b058565226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" is_pre_election: true
I20260926 13:06:43.365842 18047 raft_consensus.cc:2454] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2ad162cd5f408d9a4175b058565226 in term 0.
I20260926 13:06:43.366091 18116 leader_election.cc:302] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d; no voters:
I20260926 13:06:43.366215 17913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65dc6a0c0771435ba96ac9e2d19a2ff2" candidate_uuid: "5e2ad162cd5f408d9a4175b058565226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e7458483e784255a4e37a60b8f31bad" is_pre_election: true
I20260926 13:06:43.366261 18251 raft_consensus.cc:2781] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:06:43.366315 18251 raft_consensus.cc:482] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:06:43.366348 18251 raft_consensus.cc:3037] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:43.366376 17913 raft_consensus.cc:2454] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2ad162cd5f408d9a4175b058565226 in term 0.
I20260926 13:06:43.367089 18251 raft_consensus.cc:504] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.367228 18251 leader_election.cc:288] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [CANDIDATE]: Term 1 election: Requested vote from peers eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499), 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303)
I20260926 13:06:43.367419 18047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65dc6a0c0771435ba96ac9e2d19a2ff2" candidate_uuid: "5e2ad162cd5f408d9a4175b058565226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
I20260926 13:06:43.367475 17913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65dc6a0c0771435ba96ac9e2d19a2ff2" candidate_uuid: "5e2ad162cd5f408d9a4175b058565226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e7458483e784255a4e37a60b8f31bad"
I20260926 13:06:43.367496 18047 raft_consensus.cc:3037] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:43.367570 17913 raft_consensus.cc:3037] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:06:43.368208 18047 raft_consensus.cc:2454] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2ad162cd5f408d9a4175b058565226 in term 1.
I20260926 13:06:43.368359 17913 raft_consensus.cc:2454] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2ad162cd5f408d9a4175b058565226 in term 1.
I20260926 13:06:43.368384 18116 leader_election.cc:302] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d; no voters:
I20260926 13:06:43.368484 18251 raft_consensus.cc:2781] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:06:43.368667 18251 raft_consensus.cc:686] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2ad162cd5f408d9a4175b058565226, State: Running, Role: LEADER
I20260926 13:06:43.368780 18251 consensus_queue.cc:237] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:06:43.369709 17767 catalog_manager.cc:5495] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "5e2ad162cd5f408d9a4175b058565226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } health_report { overall_health: HEALTHY } } }
W20260926 13:06:43.374943 18228 tablet.cc:2369] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:06:43.387228 18243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.384807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.2.86.3:38343)
0926 13:06:43.384855 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 13:06:43.384857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:06:43.384889 (+ 32us) server_negotiation.cc:407] Connection header received
0926 13:06:43.384930 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:06:43.384931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:06:43.384952 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:06:43.385039 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:06:43.385191 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.385669 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.386420 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:06:43.386637 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:06:43.386713 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:06:43.386870 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:06:43.386979 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:06:43.387076 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 13:06:43.387094 (+ 18us) server_negotiation.cc:299] Negotiation successful
0926 13:06:43.387117 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:06:43.389108 17913 raft_consensus.cc:1261] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Refusing update from remote peer 5e2ad162cd5f408d9a4175b058565226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:43.389149 18047 raft_consensus.cc:1261] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Refusing update from remote peer 5e2ad162cd5f408d9a4175b058565226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:06:43.389453 18251 consensus_queue.cc:1035] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:43.389590 18254 consensus_queue.cc:1035] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:06:43.392066 18259 mvcc.cc:205] Tried to move back new op lower bound from 7333593101878673408 to 7333593101799337984. Current Snapshot: MvccSnapshot[applied={T|T < 7333593101878673408}]
I20260926 13:06:43.392056 18257 mvcc.cc:205] Tried to move back new op lower bound from 7333593101878673408 to 7333593101799337984. Current Snapshot: MvccSnapshot[applied={T|T < 7333593101878673408}]
I20260926 13:06:43.394685 17767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:43.394804 17767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:06:43.395550 17767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Sending DeleteTablet for 3 replicas of tablet 65dc6a0c0771435ba96ac9e2d19a2ff2
I20260926 13:06:43.395948 18161 tablet_service.cc:1514] Processing DeleteTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:47264
I20260926 13:06:43.395979 18027 tablet_service.cc:1514] Processing DeleteTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:45700
I20260926 13:06:43.396185 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17830
I20260926 13:06:43.396207 18267 tablet_replica.cc:330] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: stopping tablet replica
I20260926 13:06:43.396194 17893 tablet_service.cc:1514] Processing DeleteTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:46386
I20260926 13:06:43.396299 18267 raft_consensus.cc:2229] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:06:43.396396 18267 raft_consensus.cc:2258] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:43.396991 18268 tablet_replica.cc:330] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: stopping tablet replica
I20260926 13:06:43.397313 18267 ts_tablet_manager.cc:1905] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:43.398762 18268 raft_consensus.cc:2229] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:06:43.398921 18268 raft_consensus.cc:2258] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:06:43.399180 18267 ts_tablet_manager.cc:1918] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:43.399251 18267 log.cc:1196] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/65dc6a0c0771435ba96ac9e2d19a2ff2
I20260926 13:06:43.399513 18267 ts_tablet_manager.cc:1939] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting consensus metadata
I20260926 13:06:43.399940 17754 catalog_manager.cc:4841] TS eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (table test-table [id=b826609e73884d5aae23362f944e0329]) successfully deleted
I20260926 13:06:43.400151 18268 ts_tablet_manager.cc:1905] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:06:43.401594 18268 ts_tablet_manager.cc:1918] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:06:43.401691 18268 log.cc:1196] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/65dc6a0c0771435ba96ac9e2d19a2ff2
I20260926 13:06:43.401950 18268 ts_tablet_manager.cc:1939] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e2ad162cd5f408d9a4175b058565226: Deleting consensus metadata
I20260926 13:06:43.402499 17754 catalog_manager.cc:4841] TS 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (table test-table [id=b826609e73884d5aae23362f944e0329]) successfully deleted
W20260926 13:06:43.403669 17754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:33303 (error 108)
I20260926 13:06:43.403877 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17964
I20260926 13:06:43.404798 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:06:43.404664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33303 (local address 127.0.0.1:46400)
0926 13:06:43.404723 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 13:06:43.404763 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 13:06:43.404942 17754 catalog_manager.cc:4584] TS 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33303: connect: Connection refused (error 111)
I20260926 13:06:43.408530 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18098
I20260926 13:06:43.413286 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 17736
I20260926 13:07:03.418195 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--webserver_interface=127.2.86.62
--webserver_port=36277
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:03.498034 18270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:03.498234 18270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:03.498260 18270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:03.499402 18270 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:07:03.499442 18270 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:07:03.499456 18270 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:07:03.499468 18270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:03.499523 18270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:03.499542 18270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:03.499565 18270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:03.499589 18270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:03.501066 18270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=36277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:03.501312 18270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:03.501545 18270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:03.509716 18279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.509737 18277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.509729 18276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:03.510109 18270 server_base.cc:1034] running on GCE node
I20260926 13:07:03.510365 18270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:03.510612 18270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:03.511747 18270 hybrid_clock.cc:648] HybridClock initialized: now 1790428023511737 us; error 25 us; skew 500 ppm
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:03.514267 18270 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:07:03.514719 18270 webserver.cc:460] Webserver started at http://127.2.86.62:36277/ using document root <none> and password file <none>
I20260926 13:07:03.514947 18270 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:03.515024 18270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:03.516345 18270 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.517038 18286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.517210 18270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.517310 18270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "15ecdf08525b43a2921e8e9c556fae83"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "2e47202369ca86f18ce74d7aad0166bb"
server_key_iv: "573d20e9b44a4485125846e26e15bb17"
server_key_version: "encryptionkey@0"
I20260926 13:07:03.517690 18270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:03.538184 18270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:03.540185 18270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:03.540352 18270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:03.544922 18270 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33413
I20260926 13:07:03.544971 18338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33413 every 8 connection(s)
I20260926 13:07:03.545367 18270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:03.546818 18339 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:07:03.547501 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap starting.
I20260926 13:07:03.553141 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18270
I20260926 13:07:03.553474 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:33303
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=35287
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:07:03.556228 18339 log.cc:824] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:03.558501 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:07:03.558799 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap complete.
I20260926 13:07:03.561158 18339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:03.561388 18339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Initialized, Role: FOLLOWER
I20260926 13:07:03.561566 18339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:03.561628 18339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:03.561694 18339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:03.561743 18339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:07:03.562805 18339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:03.562891 18339 leader_election.cc:302] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ecdf08525b43a2921e8e9c556fae83; no voters:
I20260926 13:07:03.563107 18339 leader_election.cc:288] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 13:07:03.563141 18343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:07:03.563256 18343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 LEADER]: Becoming Leader. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Running, Role: LEADER
I20260926 13:07:03.563378 18339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:03.563401 18343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:03.563722 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:07:03.563836 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:03.563717 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ecdf08525b43a2921e8e9c556fae83. Latest consensus state: current_term: 2 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:07:03.564208 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:03.564265 18350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:03.565128 18350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:03.565234 18350 catalog_manager.cc:1259] Loaded cluster ID: 24c071ec10ec4881a4e5afb536c55c09
I20260926 13:07:03.565274 18350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:03.566164 18350 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:03.566370 18350 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Loaded TSK: 0
I20260926 13:07:03.575603 18350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Generated new TSK 1
I20260926 13:07:03.575711 18350 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:07:03.652472 18341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:03.652704 18341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:03.652745 18341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:03.653949 18341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:03.654016 18341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:03.654060 18341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:03.655534 18341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:33303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=35287
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:03.655787 18341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:03.656015 18341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:03.661075 18366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.661087 18367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.661063 18369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:03.661708 18341 server_base.cc:1034] running on GCE node
I20260926 13:07:03.661931 18341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:03.662175 18341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:03.663335 18341 hybrid_clock.cc:648] HybridClock initialized: now 1790428023663321 us; error 31 us; skew 500 ppm
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:03.666108 18341 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:07:03.666518 18341 webserver.cc:460] Webserver started at http://127.2.86.1:35287/ using document root <none> and password file <none>
I20260926 13:07:03.666764 18341 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:03.666847 18341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:03.668282 18341 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.669063 18376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.669304 18341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.669407 18341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "5e7458483e784255a4e37a60b8f31bad"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "4328da9b7e9e0c767b1fd54cd0d46c5b"
server_key_iv: "62a6f0deac9317a9ff8a5421b899b0db"
server_key_version: "encryptionkey@0"
I20260926 13:07:03.669857 18341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:03.698632 18341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:03.701184 18341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:03.701342 18341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:03.701619 18341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:03.702122 18383 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:07:03.703579 18341 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:07:03.703639 18341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 13:07:03.703671 18341 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:07:03.704471 18341 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:07:03.704514 18341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 13:07:03.704597 18383 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap starting.
I20260926 13:07:03.711104 18341 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:33303
I20260926 13:07:03.711542 18341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:07:03.713050 18490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:33303 every 8 connection(s)
I20260926 13:07:03.715128 18383 log.cc:824] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:03.718050 18383 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:07:03.718364 18383 tablet_bootstrap.cc:492] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap complete.
I20260926 13:07:03.719597 18383 ts_tablet_manager.cc:1397] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260926 13:07:03.719748 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18341
I20260926 13:07:03.720029 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:35499
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=38847
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:07:03.720965 18383 raft_consensus.cc:348] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:07:03.721342 18383 raft_consensus.cc:729] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7458483e784255a4e37a60b8f31bad, State: Initialized, Role: FOLLOWER
I20260926 13:07:03.721467 18383 consensus_queue.cc:260] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } } peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } }
I20260926 13:07:03.721716 18383 ts_tablet_manager.cc:1428] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:03.727769 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.713713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:41579)
0926 13:07:03.715448 (+ 1735us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.715500 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.716049 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.716623 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.716627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.716993 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:03.717202 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.717208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.718473 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.718474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.719313 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.719315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.720260 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.720261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.720417 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.721249 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:03.721259 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:03.723257 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:03.725874 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.725881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.725883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.726177 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.726491 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.726492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.726492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.726623 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.727129 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:03.727136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:03.727294 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.727410 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.727592 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1679,"thread_start_us":101,"threads_started":1}
I20260926 13:07:03.728107 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.713774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:41579 (local address 127.2.86.62:33413)
0926 13:07:03.716256 (+ 2482us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.716265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.716298 (+ 33us) server_negotiation.cc:407] Connection header received
0926 13:07:03.716405 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.716409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.716450 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.716594 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:03.717333 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.718290 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.719422 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.720141 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.720752 (+ 611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.723372 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:03.723376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:03.723376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:03.723388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:03.725776 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.726273 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.726276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.726277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.726409 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.726701 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.726702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.726703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.727040 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:03.727138 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.727501 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.727663 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2420,"thread_start_us":95,"threads_started":1}
I20260926 13:07:03.728840 18491 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:07:03.728974 18491 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:03.729203 18491 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:07:03.730000 18303 ts_manager.cc:194] Registered new tserver with Master: 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303)
I20260926 13:07:03.731343 18303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:41579
I20260926 13:07:03.732195 18491 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
I20260926 13:07:03.736397 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.730771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.2.86.1:33303)
0926 13:07:03.733011 (+ 2240us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.733016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.733095 (+ 79us) server_negotiation.cc:407] Connection header received
0926 13:07:03.733186 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.733188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.733200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.733274 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:03.733419 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.734084 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.735583 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.736031 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.736095 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.736130 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.736314 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.736345 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2205,"thread_start_us":68,"threads_started":1}
I20260926 13:07:03.736426 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.730699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33303 (local address 127.0.0.1:55584)
0926 13:07:03.732983 (+ 2284us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.733022 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.733101 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.733291 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.733293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.733295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:03.733357 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.733359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.734164 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.734164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.735499 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.735502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.736111 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.736112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.736201 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.736219 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.736296 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.736350 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2250,"thread_start_us":104,"threads_started":1}
I20260926 13:07:03.736819 18425 tablet_service.cc:1514] Processing DeleteTablet for tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:06:43 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:55584
I20260926 13:07:03.737406 18500 tablet_replica.cc:330] stopping tablet replica
I20260926 13:07:03.737524 18500 raft_consensus.cc:2229] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:03.737594 18500 raft_consensus.cc:2258] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:03.737874 18500 ts_tablet_manager.cc:1905] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:03.740003 18500 ts_tablet_manager.cc:1918] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:03.740085 18500 log.cc:1196] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal/wals/65dc6a0c0771435ba96ac9e2d19a2ff2
I20260926 13:07:03.740320 18500 ts_tablet_manager.cc:1939] T 65dc6a0c0771435ba96ac9e2d19a2ff2 P 5e7458483e784255a4e37a60b8f31bad: Deleting consensus metadata
I20260926 13:07:03.740803 18289 catalog_manager.cc:4841] TS 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303): tablet 65dc6a0c0771435ba96ac9e2d19a2ff2 (table test-table [id=b826609e73884d5aae23362f944e0329]) successfully deleted
W20260926 13:07:03.806450 18497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:03.806690 18497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:03.806731 18497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:03.807912 18497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:03.807981 18497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:03.808027 18497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:03.809549 18497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:35499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=38847
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:03.809820 18497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:03.810058 18497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:03.815030 18506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.815040 18508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.815048 18505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:03.815256 18497 server_base.cc:1034] running on GCE node
I20260926 13:07:03.815482 18497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:03.815791 18497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:03.817008 18497 hybrid_clock.cc:648] HybridClock initialized: now 1790428023816979 us; error 27 us; skew 500 ppm
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:03.819473 18497 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:07:03.819950 18497 webserver.cc:460] Webserver started at http://127.2.86.2:38847/ using document root <none> and password file <none>
I20260926 13:07:03.820209 18497 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:03.820294 18497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:03.821717 18497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.822294 18515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.822454 18497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:03.822562 18497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "be5e2e5d7f29d3eca9a0b662f09ed9a5"
server_key_iv: "2ad7d30482f63ee212844490125b7cbc"
server_key_version: "encryptionkey@0"
I20260926 13:07:03.822888 18497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:03.835237 18497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:03.837212 18497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:03.837369 18497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:03.837605 18497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:03.837900 18497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:03.837953 18497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.837997 18497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:03.838020 18497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.844048 18497 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:35499
I20260926 13:07:03.844095 18628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:35499 every 8 connection(s)
I20260926 13:07:03.844417 18497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:03.846201 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18497
I20260926 13:07:03.846465 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:38343
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=39635
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:03.855262 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.845169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:42591 (local address 127.2.86.62:33413)
0926 13:07:03.845259 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.845261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.845446 (+ 185us) server_negotiation.cc:407] Connection header received
0926 13:07:03.845928 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.845930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.845941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.846033 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:03.847122 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.847900 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.849344 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.849726 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.849804 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.852288 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:03.852289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:03.852289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:03.852296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:03.853802 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.854304 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.854304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.854305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.854354 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.854648 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.854648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.854648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.854741 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:03.854788 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.855132 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.855199 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 13:07:03.855577 18631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.845142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:42591)
0926 13:07:03.845321 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.845355 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.845814 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.846100 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.846103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.846808 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:03.847020 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.847026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.847995 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.847996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.849255 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.849258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.849812 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.849813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.849904 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.850450 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:03.850458 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:03.852195 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:03.853882 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.853889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.853891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.854225 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.854422 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.854423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.854424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.854578 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.854806 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:03.854812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:03.854984 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.855119 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.855297 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260926 13:07:03.856149 18629 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:07:03.856236 18629 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:03.856418 18629 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:07:03.856812 18303 ts_manager.cc:194] Registered new tserver with Master: eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
I20260926 13:07:03.857230 18303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:42591
W20260926 13:07:03.932355 18632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:03.932581 18632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:03.932617 18632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:03.933794 18632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:03.933862 18632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:03.933912 18632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:03.935354 18632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:38343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=39635
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:03.935606 18632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:03.935827 18632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:07:03.940723 18632 server_base.cc:1034] running on GCE node
W20260926 13:07:03.940659 18640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.940691 18637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:03.940666 18638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:03.941120 18632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:03.941378 18632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:03.942519 18632 hybrid_clock.cc:648] HybridClock initialized: now 1790428023942499 us; error 36 us; skew 500 ppm
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:03.944990 18632 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:07:03.945427 18632 webserver.cc:460] Webserver started at http://127.2.86.3:39635/ using document root <none> and password file <none>
I20260926 13:07:03.945622 18632 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:03.945674 18632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:03.946915 18632 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.947455 18647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.947618 18632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:03.947717 18632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "5e2ad162cd5f408d9a4175b058565226"
format_stamp: "Formatted at 2026-09-26 13:06:43 on dist-test-slave-j4lq"
server_key: "d3cabf1699f61266e9551368acd23b88"
server_key_iv: "dbb79374fc169a49236f265da10fc9ac"
server_key_version: "encryptionkey@0"
I20260926 13:07:03.948053 18632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:03.966302 18632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:03.969340 18632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:03.969502 18632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:03.969759 18632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:03.970081 18632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:03.970134 18632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.970175 18632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:03.970204 18632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:03.976198 18632 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38343
I20260926 13:07:03.976255 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38343 every 8 connection(s)
I20260926 13:07:03.976588 18632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:03.981886 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18632
Sep 26 13:07:03 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:03.985813 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.977394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:58903 (local address 127.2.86.62:33413)
0926 13:07:03.977475 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.977477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.977666 (+ 189us) server_negotiation.cc:407] Connection header received
0926 13:07:03.978142 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.978144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.978153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.978204 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:03.979045 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.979515 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.980453 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.980631 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.980678 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.982986 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:03.982988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:03.982988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:03.982995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:03.984417 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.984823 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.984824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.984824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.984919 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.985209 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.985210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.985210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.985310 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:03.985362 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.985693 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.985751 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:07:03.986107 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.977372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:58903)
0926 13:07:03.977542 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.977576 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.978062 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.978254 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.978258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.978789 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:03.978972 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.978976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.979590 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.979591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.980383 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.980385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.980697 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.980698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.980805 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.981392 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:03.981400 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:03.982895 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:03.984525 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.984531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.984533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.984750 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.984983 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:03.984984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:03.984984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:03.985131 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:03.985377 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:03.985383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:03.985549 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.985680 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.985850 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260926 13:07:03.986588 18761 heartbeater.cc:347] Connected to a master server at 127.2.86.62:33413
I20260926 13:07:03.986685 18761 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:03.986869 18761 heartbeater.cc:511] Master 127.2.86.62:33413 requested a full tablet report, sending...
I20260926 13:07:03.987268 18303 ts_manager.cc:194] Registered new tserver with Master: 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343)
I20260926 13:07:03.987767 18303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:58903
I20260926 13:07:03.988840 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.982261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.2.86.62:33413)
0926 13:07:03.982372 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.982374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.982484 (+ 110us) server_negotiation.cc:407] Connection header received
0926 13:07:03.982536 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.982537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.982549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.982614 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:03.983170 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.983645 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.984400 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.984612 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.984687 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.986034 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:03.986035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:03.986035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:03.986041 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:03.987830 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.988175 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.988176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.988176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.988226 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:03.988423 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:03.988423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:03.988424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:03.988519 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:03.988631 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.988725 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.988781 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260926 13:07:03.989436 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:03.991820 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.989796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.2.86.62:33413)
0926 13:07:03.989845 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.989847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.989932 (+ 85us) server_negotiation.cc:407] Connection header received
0926 13:07:03.989986 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.989987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.990016 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.990063 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:03.990146 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.990564 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.991220 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.991403 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.991451 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.991597 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:03.991675 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:03.991731 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.991763 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.991779 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:07:03.992100 18303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:03.993803 18303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:03.995993 18425 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc21f9df1fa403395505ed2809287f6 (DEFAULT_TABLE table=test-table [id=574fa8c313dd4713b518aaf554f10314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:03.996184 18425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc21f9df1fa403395505ed2809287f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:03.997622 18383 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap starting.
I20260926 13:07:03.997946 18383 tablet_bootstrap.cc:654] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:03.998482 18383 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: No bootstrap required, opened a new log
I20260926 13:07:03.998538 18383 ts_tablet_manager.cc:1397] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:03.998654 18383 raft_consensus.cc:348] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:03.998736 18383 raft_consensus.cc:374] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:03.998787 18383 raft_consensus.cc:729] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7458483e784255a4e37a60b8f31bad, State: Initialized, Role: FOLLOWER
I20260926 13:07:03.998835 18383 consensus_queue.cc:260] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:03.998927 18383 ts_tablet_manager.cc:1428] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:03.999790 18768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.995611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.0.0.1:42822)
0926 13:07:03.995825 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.995834 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.995873 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.996204 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.996205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.996209 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:03.996269 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.996270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.997625 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.997626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.998722 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.998725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.999552 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.999552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.999613 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.999618 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.999674 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.999715 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260926 13:07:03.999802 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.995704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.2.86.2:35499)
0926 13:07:03.995985 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.995989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.996000 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:03.996039 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.996041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.996054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.996122 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:03.996348 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.997533 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.998787 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.999473 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.999541 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.999600 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.999688 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.999725 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":77,"threads_started":1}
I20260926 13:07:04.000046 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.995494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38343 (local address 127.0.0.1:45660)
0926 13:07:03.995546 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:03.995556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:03.995593 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:03.995896 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:03.995897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:03.995899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:03.995939 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.995941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.997106 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.997107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.998200 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:03.998202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.999812 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.999813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:03.999876 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.999880 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:03.999937 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 13:07:03.999981 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:07:04.000065 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:03.995591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.2.86.3:38343)
0926 13:07:03.995744 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:03.995749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:03.995758 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:03.995795 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:03.995797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:03.995812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:03.995900 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:03.995997 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.997021 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.998286 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:03.999712 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:03.999810 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:03.999865 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:03.999951 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 13:07:03.999985 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260926 13:07:04.000496 18563 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc21f9df1fa403395505ed2809287f6 (DEFAULT_TABLE table=test-table [id=574fa8c313dd4713b518aaf554f10314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:04.000630 18695 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc21f9df1fa403395505ed2809287f6 (DEFAULT_TABLE table=test-table [id=574fa8c313dd4713b518aaf554f10314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:04.000861 18563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc21f9df1fa403395505ed2809287f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:04.000946 18695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc21f9df1fa403395505ed2809287f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:04.004478 18772 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Bootstrap starting.
I20260926 13:07:04.004742 18771 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Bootstrap starting.
I20260926 13:07:04.005227 18772 tablet_bootstrap.cc:654] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:04.005409 18771 tablet_bootstrap.cc:654] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:04.005882 18772 log.cc:824] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:04.006023 18771 log.cc:824] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:04.006692 18772 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: No bootstrap required, opened a new log
I20260926 13:07:04.006697 18771 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: No bootstrap required, opened a new log
I20260926 13:07:04.006748 18772 ts_tablet_manager.cc:1397] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:07:04.006776 18771 ts_tablet_manager.cc:1397] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:04.008299 18771 raft_consensus.cc:348] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.008332 18772 raft_consensus.cc:348] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.008461 18772 raft_consensus.cc:374] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:04.008464 18771 raft_consensus.cc:374] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:04.008482 18772 raft_consensus.cc:729] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda011b97b9241c3864a10bd7ebd0a5d, State: Initialized, Role: FOLLOWER
I20260926 13:07:04.008520 18771 raft_consensus.cc:729] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2ad162cd5f408d9a4175b058565226, State: Initialized, Role: FOLLOWER
I20260926 13:07:04.008635 18772 consensus_queue.cc:260] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.008631 18771 consensus_queue.cc:260] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.008934 18771 ts_tablet_manager.cc:1428] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:04.008946 18772 ts_tablet_manager.cc:1428] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:07:04.008950 18761 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
I20260926 13:07:04.009109 18629 heartbeater.cc:503] Master 127.2.86.62:33413 was elected leader, sending a full tablet report...
I20260926 13:07:04.024165 18496 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:04.024272 18496 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.024619 18496 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
I20260926 13:07:04.027719 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.024940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:47337 (local address 127.2.86.3:38343)
0926 13:07:04.025006 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:04.025008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:04.025019 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:04.025059 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:04.025060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:04.025071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:04.025128 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:04.025276 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.025811 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.026909 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.027379 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.027464 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.027475 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:04.027623 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 13:07:04.027660 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:07:04.027735 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.024830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38343 (local address 127.2.86.1:47337)
0926 13:07:04.024902 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:04.024915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:04.024986 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:04.025139 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:04.025140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:04.025142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:04.025192 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:04.025194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.025887 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.025888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:04.026843 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:04.026845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.027462 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.027463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:04.027534 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.027540 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 13:07:04.027606 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:07:04.027647 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 13:07:04.028016 18715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc21f9df1fa403395505ed2809287f6" candidate_uuid: "5e7458483e784255a4e37a60b8f31bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2ad162cd5f408d9a4175b058565226" is_pre_election: true
I20260926 13:07:04.028215 18715 raft_consensus.cc:2454] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7458483e784255a4e37a60b8f31bad in term 0.
I20260926 13:07:04.028323 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.025040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:51549 (local address 127.2.86.2:35499)
0926 13:07:04.025108 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:04.025110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:04.025198 (+ 88us) server_negotiation.cc:407] Connection header received
0926 13:07:04.025256 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:04.025257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:04.025267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:04.025324 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:04.025812 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.026342 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.027455 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.027920 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.028005 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.028015 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:04.028224 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 13:07:04.028259 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:07:04.028338 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.024969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.2.86.1:51549)
0926 13:07:04.025134 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:04.025142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:04.025208 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:04.025327 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:04.025328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:04.025330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:04.025733 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:04.025735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.026417 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.026418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:04.027376 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:04.027378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.028026 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.028027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:04.028106 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.028109 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:04.028215 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:07:04.028251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260926 13:07:04.028613 18379 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2ad162cd5f408d9a4175b058565226, 5e7458483e784255a4e37a60b8f31bad; no voters:
I20260926 13:07:04.028733 18496 raft_consensus.cc:2781] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:04.028795 18496 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:04.028820 18496 raft_consensus.cc:3037] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:04.028950 18583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc21f9df1fa403395505ed2809287f6" candidate_uuid: "5e7458483e784255a4e37a60b8f31bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" is_pre_election: true
I20260926 13:07:04.029083 18583 raft_consensus.cc:2454] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7458483e784255a4e37a60b8f31bad in term 0.
I20260926 13:07:04.029661 18496 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.029784 18496 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 1 election: Requested vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
I20260926 13:07:04.029930 18715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc21f9df1fa403395505ed2809287f6" candidate_uuid: "5e7458483e784255a4e37a60b8f31bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2ad162cd5f408d9a4175b058565226"
I20260926 13:07:04.029990 18583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc21f9df1fa403395505ed2809287f6" candidate_uuid: "5e7458483e784255a4e37a60b8f31bad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
I20260926 13:07:04.030030 18715 raft_consensus.cc:3037] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:04.030077 18583 raft_consensus.cc:3037] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:04.030706 18715 raft_consensus.cc:2454] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7458483e784255a4e37a60b8f31bad in term 1.
I20260926 13:07:04.030740 18583 raft_consensus.cc:2454] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7458483e784255a4e37a60b8f31bad in term 1.
I20260926 13:07:04.030892 18379 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2ad162cd5f408d9a4175b058565226, 5e7458483e784255a4e37a60b8f31bad; no voters:
I20260926 13:07:04.030994 18496 raft_consensus.cc:2781] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:04.031176 18496 raft_consensus.cc:686] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 LEADER]: Becoming Leader. State: Replica: 5e7458483e784255a4e37a60b8f31bad, State: Running, Role: LEADER
I20260926 13:07:04.031257 18496 consensus_queue.cc:237] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:04.031960 18303 catalog_manager.cc:5495] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e7458483e784255a4e37a60b8f31bad (127.2.86.1). New cstate: current_term: 1 leader_uuid: "5e7458483e784255a4e37a60b8f31bad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } health_report { overall_health: HEALTHY } } }
I20260926 13:07:04.039006 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.036727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.2.86.1:33303)
0926 13:07:04.036783 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:04.036785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:04.036793 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:07:04.036845 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:04.036846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:04.036928 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:04.037010 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:04.037150 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.037594 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.038332 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.038521 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.038594 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.038759 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:04.038798 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 13:07:04.038934 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55590: token expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 13:07:04.039130 18499 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55590: token expired
I20260926 13:07:04.045173 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.039388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.2.86.62:33413)
0926 13:07:04.039450 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:04.039452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:04.039461 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:04.039485 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:04.039486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:04.039495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:04.039560 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:04.040042 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.040493 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.041258 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.041457 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.041539 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.042698 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:04.042700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:04.042700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:04.042707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:04.044001 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:04.044358 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:04.044360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:04.044360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:04.044421 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:04.044616 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:04.044617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:04.044618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:04.044752 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:04.044859 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:04.044999 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 13:07:04.045075 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:07:04.047843 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.045995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.2.86.1:33303)
0926 13:07:04.046030 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:04.046032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:04.046080 (+ 48us) server_negotiation.cc:407] Connection header received
0926 13:07:04.046099 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:04.046100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:04.046108 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:04.046154 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:04.046231 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.046636 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.047268 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:04.047449 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:04.047512 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:04.047612 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:04.047688 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:04.047726 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:04.047783 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 13:07:04.047798 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:07:04.049314 18583 raft_consensus.cc:1261] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Refusing update from remote peer 5e7458483e784255a4e37a60b8f31bad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:04.049388 18715 raft_consensus.cc:1261] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Refusing update from remote peer 5e7458483e784255a4e37a60b8f31bad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:04.049679 18778 consensus_queue.cc:1035] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:04.049808 18496 consensus_queue.cc:1035] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:04.052225 18783 mvcc.cc:205] Tried to move back new op lower bound from 7333593186503634944 to 7333593186432581632. Current Snapshot: MvccSnapshot[applied={T|T < 7333593186503634944}]
I20260926 13:07:04.052330 18782 mvcc.cc:205] Tried to move back new op lower bound from 7333593186503634944 to 7333593186432581632. Current Snapshot: MvccSnapshot[applied={T|T < 7333593186503634944}]
I20260926 13:07:04.055114 18301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:04.055207 18301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:04.056371 18301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Sending DeleteTablet for 3 replicas of tablet 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:04.056648 18695 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fc21f9df1fa403395505ed2809287f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:04 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:45660
I20260926 13:07:04.056703 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18341
I20260926 13:07:04.056759 18563 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fc21f9df1fa403395505ed2809287f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:04 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:42822
I20260926 13:07:04.061012 18792 tablet_replica.cc:330] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: stopping tablet replica
I20260926 13:07:04.061105 18791 tablet_replica.cc:330] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: stopping tablet replica
I20260926 13:07:04.061120 18792 raft_consensus.cc:2229] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:04.061219 18792 raft_consensus.cc:2258] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:04.061215 18791 raft_consensus.cc:2229] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:04.061316 18791 raft_consensus.cc:2258] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:04.061703 18791 ts_tablet_manager.cc:1905] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:04.061708 18792 ts_tablet_manager.cc:1905] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:07:04.062490 18289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:33303 (error 108)
I20260926 13:07:04.062940 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.062815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33303 (local address 127.0.0.1:55618)
0926 13:07:04.062860 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:04.062890 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:07:04.063040 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18497
W20260926 13:07:04.063077 18289 catalog_manager.cc:4584] TS 5e7458483e784255a4e37a60b8f31bad (127.2.86.1:33303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fc21f9df1fa403395505ed2809287f6: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33303: connect: Connection refused (error 111)
I20260926 13:07:04.065048 18791 ts_tablet_manager.cc:1918] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:04.065119 18791 log.cc:1196] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:04.065375 18791 ts_tablet_manager.cc:1939] T 1fc21f9df1fa403395505ed2809287f6 P 5e2ad162cd5f408d9a4175b058565226: Deleting consensus metadata
I20260926 13:07:04.065827 18289 catalog_manager.cc:4841] TS 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): tablet 1fc21f9df1fa403395505ed2809287f6 (table test-table [id=574fa8c313dd4713b518aaf554f10314]) successfully deleted
I20260926 13:07:04.068451 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18632
I20260926 13:07:04.068609 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:04.068475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.0.0.1:42832)
0926 13:07:04.068541 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:04.068573 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.2:35499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:07:04.073424 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18270
I20260926 13:07:24.078931 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--webserver_interface=127.2.86.62
--webserver_port=36277
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:24.159479 18794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:24.159675 18794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:24.159698 18794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:24.160806 18794 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 13:07:24.160843 18794 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 13:07:24.160856 18794 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 13:07:24.160919 18794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:24.160939 18794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:24.160964 18794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:24.160996 18794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:24.161012 18794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:24.162456 18794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=36277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:24.162695 18794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:24.162906 18794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:24.167899 18799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.167900 18800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.168072 18794 server_base.cc:1034] running on GCE node
W20260926 13:07:24.167912 18802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.168453 18794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:24.168763 18794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:24.169925 18794 hybrid_clock.cc:648] HybridClock initialized: now 1790428044169918 us; error 38 us; skew 500 ppm
Sep 26 13:07:24 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:24.172219 18794 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:07:24.172673 18794 webserver.cc:460] Webserver started at http://127.2.86.62:36277/ using document root <none> and password file <none>
I20260926 13:07:24.172947 18794 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:24.173023 18794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:24.174314 18794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:07:24.174908 18809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.175062 18794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:24.175155 18794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "15ecdf08525b43a2921e8e9c556fae83"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "2e47202369ca86f18ce74d7aad0166bb"
server_key_iv: "573d20e9b44a4485125846e26e15bb17"
server_key_version: "encryptionkey@0"
I20260926 13:07:24.175485 18794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:24.181834 18794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:24.183627 18794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:24.183781 18794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:24.187470 18794 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33413
I20260926 13:07:24.187508 18861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33413 every 8 connection(s)
I20260926 13:07:24.187882 18794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:24.189303 18862 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:07:24.189929 18862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap starting.
I20260926 13:07:24.193521 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18794
I20260926 13:07:24.193783 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:33303
--local_ip_for_outbound_sockets=127.2.86.1
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=35287
--webserver_interface=127.2.86.1
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:07:24.198714 18862 log.cc:824] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:24.200925 18862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:07:24.201249 18862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Bootstrap complete.
I20260926 13:07:24.203431 18862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:24.203644 18862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Initialized, Role: FOLLOWER
I20260926 13:07:24.203799 18862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:24.203876 18862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:24.203927 18862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:24.203984 18862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 2 FOLLOWER]: Advancing to term 3
I20260926 13:07:24.205088 18862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:24.205175 18862 leader_election.cc:302] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ecdf08525b43a2921e8e9c556fae83; no voters:
I20260926 13:07:24.205392 18862 leader_election.cc:288] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 13:07:24.205425 18866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 13:07:24.205547 18866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [term 3 LEADER]: Becoming Leader. State: Replica: 15ecdf08525b43a2921e8e9c556fae83, State: Running, Role: LEADER
I20260926 13:07:24.205631 18866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } }
I20260926 13:07:24.205658 18862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:24.205896 18866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ecdf08525b43a2921e8e9c556fae83. Latest consensus state: current_term: 3 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:07:24.205964 18866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:24.206394 18868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "15ecdf08525b43a2921e8e9c556fae83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ecdf08525b43a2921e8e9c556fae83" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33413 } } }
I20260926 13:07:24.206490 18868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:24.206453 18870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:24.207298 18870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:24.207412 18870 catalog_manager.cc:1259] Loaded cluster ID: 24c071ec10ec4881a4e5afb536c55c09
I20260926 13:07:24.207439 18870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:24.208559 18870 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:24.208731 18870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Loaded TSK: 0
I20260926 13:07:24.208779 18870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15ecdf08525b43a2921e8e9c556fae83: Loaded TSK: 1
F20260926 13:07:24.209232 18870 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff8c50d99ad google::LogMessage::Fail() at ??:0
@ 0x7ff8c50dc88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff8c50d94de google::LogMessage::Flush() at ??:0
@ 0x7ff8c50daa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff8c7846104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff8c7843d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff8cd0331cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff8cd03348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff8cd033594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff8ccff9c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff8cd003ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff8c6fa70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff8c6f9c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff8c7e006db start_thread at ??:0
@ 0x7ff8c3bd761f clone at ??:0
W20260926 13:07:24.286458 18864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:24.286705 18864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:24.286763 18864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:24.287967 18864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:24.288036 18864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:24.288081 18864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:24.289608 18864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:33303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=35287
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:24.289848 18864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:24.290063 18864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:24.294920 18906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.294899 18903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.294886 18904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.295308 18864 server_base.cc:1034] running on GCE node
I20260926 13:07:24.295506 18864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:24.295782 18864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:24.296952 18864 hybrid_clock.cc:648] HybridClock initialized: now 1790428044296929 us; error 39 us; skew 500 ppm
Sep 26 13:07:24 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:24.304332 18864 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:07:24.304912 18864 webserver.cc:460] Webserver started at http://127.2.86.1:35287/ using document root <none> and password file <none>
I20260926 13:07:24.305195 18864 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:24.305267 18864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:24.306977 18864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:24.307792 18913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.308008 18864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:24.308106 18864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "5e7458483e784255a4e37a60b8f31bad"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "4328da9b7e9e0c767b1fd54cd0d46c5b"
server_key_iv: "62a6f0deac9317a9ff8a5421b899b0db"
server_key_version: "encryptionkey@0"
I20260926 13:07:24.308543 18864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:24.333556 18864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:24.336077 18864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:24.336232 18864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:24.336488 18864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:24.337085 18920 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 13:07:24.338512 18864 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 13:07:24.338579 18864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 13:07:24.338613 18864 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 13:07:24.339589 18864 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 13:07:24.339634 18864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 13:07:24.339709 18920 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap starting.
I20260926 13:07:24.346793 18864 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:33303
I20260926 13:07:24.347231 18864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:07:24.349365 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 18864
I20260926 13:07:24.349727 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:35499
--local_ip_for_outbound_sockets=127.2.86.2
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=38847
--webserver_interface=127.2.86.2
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:07:24.350304 18920 log.cc:824] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:24.350580 19027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:33303 every 8 connection(s)
I20260926 13:07:24.352571 18920 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:07:24.352916 18920 tablet_bootstrap.cc:492] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Bootstrap complete.
I20260926 13:07:24.354084 18920 ts_tablet_manager.cc:1397] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.005s
I20260926 13:07:24.355911 18920 raft_consensus.cc:348] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:24.356317 18920 raft_consensus.cc:729] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7458483e784255a4e37a60b8f31bad, State: Initialized, Role: FOLLOWER
I20260926 13:07:24.356498 18920 consensus_queue.cc:260] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:24.356807 18920 ts_tablet_manager.cc:1428] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260926 13:07:24.414402 19030 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:07:24.347937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:41391)
0926 13:07:24.348157 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.348217 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:24.349044 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:24.414203 (+ 65159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
W20260926 13:07:24.414677 19028 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33413 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:33413: Client connection negotiation failed: client connection to 127.2.86.62:33413: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:07:24.415050 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.414936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:53911)
0926 13:07:24.414988 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.415021 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 13:07:24.415397 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.415316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:52583)
0926 13:07:24.415359 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.415382 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 13:07:24.415519 19028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 13:07:24.434813 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:24.434994 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:24.435019 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:24.436216 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:24.436256 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:24.436283 19032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:24.437788 19032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:35499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=38847
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:24.438009 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:24.438223 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:24.443015 19039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.443015 19041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.443162 19038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.443178 19032 server_base.cc:1034] running on GCE node
I20260926 13:07:24.443414 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:24.443615 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:24.444789 19032 hybrid_clock.cc:648] HybridClock initialized: now 1790428044444777 us; error 27 us; skew 500 ppm
Sep 26 13:07:24 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:24.447288 19032 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:07:24.447727 19032 webserver.cc:460] Webserver started at http://127.2.86.2:38847/ using document root <none> and password file <none>
I20260926 13:07:24.447933 19032 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:24.447976 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:24.449321 19032 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:07:24.449942 19048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.450148 19032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:24.450230 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "eda011b97b9241c3864a10bd7ebd0a5d"
format_stamp: "Formatted at 2026-09-26 13:06:42 on dist-test-slave-j4lq"
server_key: "be5e2e5d7f29d3eca9a0b662f09ed9a5"
server_key_iv: "2ad7d30482f63ee212844490125b7cbc"
server_key_version: "encryptionkey@0"
I20260926 13:07:24.450537 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:24.457551 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:24.459887 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:24.460047 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:24.460261 19032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:24.460667 19055 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 13:07:24.462033 19055 ts_tablet_manager.cc:1852] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 13:07:24.462306 19055 ts_tablet_manager.cc:1905] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:24.464607 19055 ts_tablet_manager.cc:1918] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:24.464709 19055 log.cc:1196] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:24.465067 19055 ts_tablet_manager.cc:1939] T 1fc21f9df1fa403395505ed2809287f6 P eda011b97b9241c3864a10bd7ebd0a5d: Deleting consensus metadata
I20260926 13:07:24.465284 19032 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 13:07:24.465328 19032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260926 13:07:24.465371 19032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:24.465399 19032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.471594 19032 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:35499
I20260926 13:07:24.471655 19162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:35499 every 8 connection(s)
I20260926 13:07:24.471930 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:24.472981 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.472571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:52159)
0926 13:07:24.472727 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.472791 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
W20260926 13:07:24.473161 19163 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33413 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:33413: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
I20260926 13:07:24.473439 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.473359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:41265)
0926 13:07:24.473400 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.473424 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:07:24.473665 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.473611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:55819)
0926 13:07:24.473634 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.473654 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260926 13:07:24.473747 19163 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:07:24.475981 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19032
I20260926 13:07:24.476254 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:38343
--local_ip_for_outbound_sockets=127.2.86.3
--tserver_master_addrs=127.2.86.62:33413
--webserver_port=39635
--webserver_interface=127.2.86.3
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:24.556504 19166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:24.556739 19166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:24.556779 19166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:24.557981 19166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:24.558048 19166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:24.558094 19166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:24.559520 19166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:38343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=39635
--tserver_master_addrs=127.2.86.62:33413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:24.559792 19166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:24.560019 19166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:24.564805 19171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:24.564800 19174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.564831 19166 server_base.cc:1034] running on GCE node
W20260926 13:07:24.564797 19172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:24.565231 19166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:24.565449 19166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:24.566596 19166 hybrid_clock.cc:648] HybridClock initialized: now 1790428044566588 us; error 26 us; skew 500 ppm
Sep 26 13:07:24 dist-test-slave-j4lq krb5kdc[17723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:24.568975 19166 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:07:24.569417 19166 webserver.cc:460] Webserver started at http://127.2.86.3:39635/ using document root <none> and password file <none>
I20260926 13:07:24.569629 19166 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:24.569697 19166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:24.570922 19166 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:24.571480 19181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.571642 19166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:24.571736 19166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "5e2ad162cd5f408d9a4175b058565226"
format_stamp: "Formatted at 2026-09-26 13:06:43 on dist-test-slave-j4lq"
server_key: "d3cabf1699f61266e9551368acd23b88"
server_key_iv: "dbb79374fc169a49236f265da10fc9ac"
server_key_version: "encryptionkey@0"
I20260926 13:07:24.572069 19166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:24.600543 19166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:24.602548 19166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:24.602715 19166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:24.602953 19166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:24.603266 19166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:24.603320 19166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.603363 19166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:24.603396 19166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:24.609229 19166 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:38343
I20260926 13:07:24.609274 19294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:38343 every 8 connection(s)
I20260926 13:07:24.609598 19166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:24.610556 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.610213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:34081)
0926 13:07:24.610348 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.610422 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260926 13:07:24.610982 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19166
W20260926 13:07:24.611076 19295 heartbeater.cc:644] Failed to heartbeat to 127.2.86.62:33413 (0 consecutive failures): Network error: Failed to ping master at 127.2.86.62:33413: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
I20260926 13:07:24.611385 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.611275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:49025)
0926 13:07:24.611341 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.611369 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:07:24.611737 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:24.611683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:34391)
0926 13:07:24.611704 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:24.611723 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260926 13:07:24.611853 19295 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 13:07:25.416055 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.415835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:57469)
0926 13:07:25.415953 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:25.415991 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260926 13:07:25.474370 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.474142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:45709)
0926 13:07:25.474298 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:25.474334 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260926 13:07:25.612310 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.612129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:47039)
0926 13:07:25.612233 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:25.612265 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260926 13:07:25.752300 19302 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:25.752419 19302 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:25.752717 19302 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
Sep 26 13:07:25 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.3@KRBTEST.COM
Sep 26 13:07:25 dist-test-slave-j4lq krb5kdc[17723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428044, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.2@KRBTEST.COM
I20260926 13:07:25.762643 19304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.753001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35499 (local address 127.2.86.1:60837)
0926 13:07:25.753132 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:25.753141 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:25.753180 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:25.753947 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:25.753948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:25.754266 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:25.754464 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:25.754470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.755532 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.755533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:25.756297 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:25.756299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.756902 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:25.757028 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:25.757653 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:25.757660 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:25.759460 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:25.761231 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:25.761232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:25.761473 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:25.761720 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:25.761721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:25.761794 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:25.762156 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:25.762164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:25.762294 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 13:07:25.762407 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 13:07:25.762572 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":57,"thread_start_us":80,"threads_started":1}
I20260926 13:07:25.762739 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.752978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:37417 (local address 127.2.86.3:38343)
0926 13:07:25.753162 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:25.753172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:25.753199 (+ 27us) server_negotiation.cc:407] Connection header received
0926 13:07:25.753632 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:25.753634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:25.753674 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:25.753827 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:25.754584 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.755418 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756401 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.756785 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756959 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:25.759178 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:25.759181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:25.759181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:25.759190 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:25.761052 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761490 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:25.761492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:25.761493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:25.761602 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761863 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:25.761864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:25.761865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:25.762090 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:25.762181 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:25.762477 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 13:07:25.762625 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260926 13:07:25.762890 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.752859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:38343 (local address 127.2.86.1:37417)
0926 13:07:25.752945 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:25.752959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:25.753024 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:25.753854 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:25.753857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:25.754274 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:25.754464 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:25.754470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.755520 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.755521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:25.756298 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:25.756299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.756905 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:25.757028 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:25.757602 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:25.757610 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:25.759076 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:25.761128 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:25.761137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:25.761416 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:25.761676 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:25.761678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:25.761794 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:25.762181 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:25.762182 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:25.762294 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 13:07:25.762407 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 13:07:25.762658 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":121}
I20260926 13:07:25.763054 19305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:25.753142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:60837 (local address 127.2.86.2:35499)
0926 13:07:25.753283 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:25.753293 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:25.753313 (+ 20us) server_negotiation.cc:407] Connection header received
0926 13:07:25.753777 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:25.753779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:25.753816 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:25.753953 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:25.754581 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.755420 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756398 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:25.756785 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:25.756959 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:25.759557 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:25.759560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:25.759561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:25.759577 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:25.761149 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761548 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:25.761551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:25.761551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:25.761625 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:25.761871 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:25.761872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:25.761873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:25.762088 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:25.762178 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:25.762821 (+ 643us) server_negotiation.cc:299] Negotiation successful
0926 13:07:25.762962 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
W20260926 13:07:25.763808 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
W20260926 13:07:25.763923 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:25.763962 18916 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e7458483e784255a4e37a60b8f31bad; no voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d
I20260926 13:07:25.764056 19302 raft_consensus.cc:2726] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:07:26.416729 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:26.416519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:53645)
0926 13:07:26.416643 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:26.416682 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260926 13:07:26.474911 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:26.474741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:40595)
0926 13:07:26.474845 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:26.474879 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260926 13:07:26.613006 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:26.612743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:53865)
0926 13:07:26.612939 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:26.612972 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260926 13:07:27.332248 19309 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:27.332368 19309 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:27.332576 19309 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
W20260926 13:07:27.333076 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
W20260926 13:07:27.333154 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:27.333170 18916 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e7458483e784255a4e37a60b8f31bad; no voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d
I20260926 13:07:27.333277 19309 raft_consensus.cc:2726] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:07:27.417351 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:27.417164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:41661)
0926 13:07:27.417276 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:27.417307 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260926 13:07:27.475559 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:27.475352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:48213)
0926 13:07:27.475452 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:27.475481 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260926 13:07:27.613564 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:27.613385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:51773)
0926 13:07:27.613492 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:27.613525 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260926 13:07:28.417975 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:28.417749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:59429)
0926 13:07:28.417890 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:28.417924 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
I20260926 13:07:28.476162 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:28.475983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:48937)
0926 13:07:28.476093 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:28.476125 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260926 13:07:28.614244 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:28.614054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:48943)
0926 13:07:28.614171 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:28.614208 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260926 13:07:29.418635 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:29.418438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:59421)
0926 13:07:29.418555 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:29.418592 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260926 13:07:29.476805 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:29.476606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:51073)
0926 13:07:29.476727 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:29.476766 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260926 13:07:29.614854 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:29.614659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:33975)
0926 13:07:29.614777 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:29.614813 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260926 13:07:30.198892 19319 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:30.198983 19319 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:30.199190 19319 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
W20260926 13:07:30.199649 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
W20260926 13:07:30.199728 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:30.199757 18916 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e7458483e784255a4e37a60b8f31bad; no voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d
I20260926 13:07:30.199812 19319 raft_consensus.cc:2726] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:07:30.419265 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:30.419063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:57031)
0926 13:07:30.419186 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:30.419218 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260926 13:07:30.477522 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:30.477337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:43013)
0926 13:07:30.477449 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:30.477484 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260926 13:07:30.615574 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:30.615359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:36061)
0926 13:07:30.615500 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:30.615533 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260926 13:07:31.419932 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:31.419730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:59043)
0926 13:07:31.419851 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:31.419887 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260926 13:07:31.478149 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:31.477962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:46479)
0926 13:07:31.478075 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:31.478107 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260926 13:07:31.616214 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:31.616009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:60913)
0926 13:07:31.616141 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:31.616176 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260926 13:07:32.420598 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:32.420371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:47067)
0926 13:07:32.420517 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:32.420552 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260926 13:07:32.478780 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:32.478593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:58421)
0926 13:07:32.478708 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:32.478741 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260926 13:07:32.616861 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:32.616676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:43737)
0926 13:07:32.616788 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:32.616823 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260926 13:07:33.421290 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:33.421087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:47055)
0926 13:07:33.421199 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:33.421234 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260926 13:07:33.479346 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:33.479175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:38331)
0926 13:07:33.479283 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:33.479312 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260926 13:07:33.617499 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:33.617314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:49539)
0926 13:07:33.617424 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:33.617459 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260926 13:07:34.257052 19332 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:34.257144 19332 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:34.257329 19332 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
W20260926 13:07:34.257747 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
W20260926 13:07:34.257827 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:34.257844 18916 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e7458483e784255a4e37a60b8f31bad; no voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d
I20260926 13:07:34.257915 19332 raft_consensus.cc:2726] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:07:34.421964 19333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:34.421754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:59327)
0926 13:07:34.421884 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:34.421918 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260926 13:07:34.479956 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:34.479741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:59411)
0926 13:07:34.479875 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:34.479912 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":56,"threads_started":1}
I20260926 13:07:34.618191 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:34.617950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:47669)
0926 13:07:34.618112 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:34.618149 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260926 13:07:35.422655 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:35.422440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:46413)
0926 13:07:35.422575 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:35.422611 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260926 13:07:35.480630 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:35.480411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:40187)
0926 13:07:35.480549 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:35.480586 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260926 13:07:35.618866 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:35.618677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:37867)
0926 13:07:35.618792 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:35.618822 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260926 13:07:36.423403 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:36.423173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:49045)
0926 13:07:36.423326 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:36.423355 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260926 13:07:36.481381 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:36.481176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:37671)
0926 13:07:36.481305 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:36.481342 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260926 13:07:36.619513 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:36.619297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:52359)
0926 13:07:36.619433 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:36.619471 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260926 13:07:37.424294 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:37.424045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:40227)
0926 13:07:37.424204 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:37.424241 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260926 13:07:37.482023 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:37.481814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:46147)
0926 13:07:37.481950 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:37.481982 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260926 13:07:37.620200 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:37.619998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:41027)
0926 13:07:37.620123 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:37.620159 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260926 13:07:38.122570 19345 raft_consensus.cc:482] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:38.122663 19345 raft_consensus.cc:504] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2ad162cd5f408d9a4175b058565226" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 38343 } } peers { permanent_uuid: "eda011b97b9241c3864a10bd7ebd0a5d" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35499 } } peers { permanent_uuid: "5e7458483e784255a4e37a60b8f31bad" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33303 } }
I20260926 13:07:38.122821 19345 leader_election.cc:288] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343), eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499)
W20260926 13:07:38.123256 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eda011b97b9241c3864a10bd7ebd0a5d (127.2.86.2:35499): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
W20260926 13:07:38.123339 18916 leader_election.cc:341] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e2ad162cd5f408d9a4175b058565226 (127.2.86.3:38343): Not found: Tablet not found: 1fc21f9df1fa403395505ed2809287f6
I20260926 13:07:38.123355 18916 leader_election.cc:302] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e7458483e784255a4e37a60b8f31bad; no voters: 5e2ad162cd5f408d9a4175b058565226, eda011b97b9241c3864a10bd7ebd0a5d
I20260926 13:07:38.123436 19345 raft_consensus.cc:2726] T 1fc21f9df1fa403395505ed2809287f6 P 5e7458483e784255a4e37a60b8f31bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:07:38.425096 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:38.424802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:33091)
0926 13:07:38.425008 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:38.425050 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":128,"threads_started":1}
I20260926 13:07:38.482707 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:38.482512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:38779)
0926 13:07:38.482632 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:38.482666 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":53,"threads_started":1}
I20260926 13:07:38.620988 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:38.620682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:42599)
0926 13:07:38.620830 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:38.620927 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260926 13:07:39.425807 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:39.425618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.1:56529)
0926 13:07:39.425733 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:39.425763 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260926 13:07:39.483337 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:39.483152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.2:59603)
0926 13:07:39.483269 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:39.483299 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":52,"threads_started":1}
I20260926 13:07:39.617285 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 18864
I20260926 13:07:39.621713 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:39.621526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:33413 (local address 127.2.86.3:42217)
0926 13:07:39.621640 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:39.621676 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.62:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260926 13:07:39.622170 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19032
I20260926 13:07:39.626401 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19166
2026-09-26T13:07:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58355 ms)
[----------] 30 tests from SecurityITest (233129 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:39 dist-test-slave-j4lq krb5kdc[19354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:39 dist-test-slave-j4lq krb5kdc[19354](info): set up 2 sockets
Sep 26 13:07:39 dist-test-slave-j4lq krb5kdc[19354](info): commencing operation
krb5kdc: starting...
W20260926 13:07:40.664494 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:40 dist-test-slave-j4lq krb5kdc[19354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:07:40.780400 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:34193
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:34193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:40.861816 19367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:40.862067 19367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:40.862110 19367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:40.863273 19367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:40.863343 19367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:40.863366 19367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:40.863389 19367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:40.863416 19367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:40.864914 19367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:34193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:40.865201 19367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:40.865442 19367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:40.870538 19372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:40.870609 19367 server_base.cc:1034] running on GCE node
W20260926 13:07:40.870612 19375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:40.870574 19373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:40.871040 19367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:40.871338 19367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:40.872500 19367 hybrid_clock.cc:648] HybridClock initialized: now 1790428060872492 us; error 35 us; skew 500 ppm
Sep 26 13:07:40 dist-test-slave-j4lq krb5kdc[19354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428060, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:40.875380 19367 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:07:40.875854 19367 webserver.cc:460] Webserver started at http://127.2.86.62:41449/ using document root <none> and password file <none>
I20260926 13:07:40.876111 19367 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:40.876180 19367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:40.876331 19367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:40.877539 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "ff1c63bba1d94384bfef4a9d3512272f"
format_stamp: "Formatted at 2026-09-26 13:07:40 on dist-test-slave-j4lq"
server_key: "98bc1ade54ec713578cec4a06f21a978"
server_key_iv: "b6e20a954f63499ff189022110282b35"
server_key_version: "encryptionkey@0"
I20260926 13:07:40.877898 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "ff1c63bba1d94384bfef4a9d3512272f"
format_stamp: "Formatted at 2026-09-26 13:07:40 on dist-test-slave-j4lq"
server_key: "98bc1ade54ec713578cec4a06f21a978"
server_key_iv: "b6e20a954f63499ff189022110282b35"
server_key_version: "encryptionkey@0"
I20260926 13:07:40.879273 19367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:40.880034 19382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:40.880215 19367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:40.880326 19367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "ff1c63bba1d94384bfef4a9d3512272f"
format_stamp: "Formatted at 2026-09-26 13:07:40 on dist-test-slave-j4lq"
server_key: "98bc1ade54ec713578cec4a06f21a978"
server_key_iv: "b6e20a954f63499ff189022110282b35"
server_key_version: "encryptionkey@0"
I20260926 13:07:40.880399 19367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:40.897538 19367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:40.899847 19367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:40.900144 19367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:40.904208 19367 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:34193
I20260926 13:07:40.904243 19434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:34193 every 8 connection(s)
I20260926 13:07:40.904623 19367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:40.905402 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19367
I20260926 13:07:40.905517 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:40.905579 19435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:40.905771 2392 external_mini_cluster.cc:1428] Setting key b29630f47ec65b1f52e4ee8a450b8352
I20260926 13:07:40.908717 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: Bootstrap starting.
I20260926 13:07:40.909538 19435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:40.909829 19435 log.cc:824] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:40.910567 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: No bootstrap required, opened a new log
Sep 26 13:07:40 dist-test-slave-j4lq krb5kdc[19354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:40.912909 19435 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } }
I20260926 13:07:40.913103 19435 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:40.913157 19435 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1c63bba1d94384bfef4a9d3512272f, State: Initialized, Role: FOLLOWER
I20260926 13:07:40.913337 19435 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } }
I20260926 13:07:40.913439 19435 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:40.913506 19435 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:40.913571 19435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:40.914326 19435 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } }
I20260926 13:07:40.914463 19435 leader_election.cc:302] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff1c63bba1d94384bfef4a9d3512272f; no voters:
I20260926 13:07:40.914687 19435 leader_election.cc:288] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:40.914741 19440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:40.914958 19440 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [term 1 LEADER]: Becoming Leader. State: Replica: ff1c63bba1d94384bfef4a9d3512272f, State: Running, Role: LEADER
I20260926 13:07:40.915015 19435 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:40.916303 19440 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } }
I20260926 13:07:40.916922 19441 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff1c63bba1d94384bfef4a9d3512272f. Latest consensus state: current_term: 1 leader_uuid: "ff1c63bba1d94384bfef4a9d3512272f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } } }
I20260926 13:07:40.916953 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff1c63bba1d94384bfef4a9d3512272f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff1c63bba1d94384bfef4a9d3512272f" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 34193 } } }
I20260926 13:07:40.917035 19441 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:40.917055 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f [sys.catalog]: This master's current role is: LEADER
W20260926 13:07:40.917759 19454 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:07:40.917814 19454 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:07:40.917845 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:40.906305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.2.86.62:34193)
0926 13:07:40.906498 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:40.906515 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:40.906541 (+ 26us) server_negotiation.cc:407] Connection header received
0926 13:07:40.907017 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:40.907024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:40.907073 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:40.907188 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:40.908009 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:40.909041 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:40.910111 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:40.910632 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:40.910734 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:40.913480 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:40.913485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:40.913485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:40.913499 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:40.915741 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:40.916163 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:40.916166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:40.916167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:40.916275 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:40.916481 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:40.916482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:40.916483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:40.916847 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:40.917509 (+ 662us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:40.917569 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 13:07:40.917674 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260926 13:07:40.917867 19455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:40.918365 19455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:40.919970 19455 catalog_manager.cc:1347] Generated new cluster ID: dd1b1b216a8e483cb65531588c16ae86
I20260926 13:07:40.920034 19455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:40.929401 19455 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:40.930158 19455 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:40.934599 19455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: Generated new TSK 0
I20260926 13:07:40.934804 19455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:07:40.964437 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:41.046079 19461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:41.046289 19461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:41.046314 19461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:41.047396 19461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:41.047428 19461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:41.047454 19461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:41.048969 19461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:41.049142 19461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:41.049309 19461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:41.054062 19469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.054214 19461 server_base.cc:1034] running on GCE node
W20260926 13:07:41.054143 19466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:41.054064 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.054520 19461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:41.054770 19461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:41.055913 19461 hybrid_clock.cc:648] HybridClock initialized: now 1790428061055905 us; error 31 us; skew 500 ppm
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:41.058297 19461 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:07:41.058712 19461 webserver.cc:460] Webserver started at http://127.2.86.1:35527/ using document root <none> and password file <none>
I20260926 13:07:41.058933 19461 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:41.058998 19461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:41.059149 19461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:41.060178 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "3da78569b09044dcb932b876bbe85580"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "68ef61189c51836777a3538db69e697c"
server_key_iv: "4deb0da00e955e4e7d790b881846140e"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.060586 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "3da78569b09044dcb932b876bbe85580"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "68ef61189c51836777a3538db69e697c"
server_key_iv: "4deb0da00e955e4e7d790b881846140e"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.061928 19461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 13:07:41.062626 19476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.062790 19461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:41.062888 19461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "3da78569b09044dcb932b876bbe85580"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "68ef61189c51836777a3538db69e697c"
server_key_iv: "4deb0da00e955e4e7d790b881846140e"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.062964 19461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:41.083542 19461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:41.085594 19461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:41.085763 19461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:41.085999 19461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:41.086306 19461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:41.086359 19461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.086401 19461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:41.086429 19461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.092110 19461 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:33003
I20260926 13:07:41.092190 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:33003 every 8 connection(s)
I20260926 13:07:41.092511 19461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:41.099491 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19461
I20260926 13:07:41.099613 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:41.099845 2392 external_mini_cluster.cc:1428] Setting key 42c54b32b67ba94d5d8979a79cb44356
I20260926 13:07:41.101643 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.093219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:34193 (local address 127.2.86.1:33791)
0926 13:07:41.093398 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.093439 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.093909 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.094114 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.094117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.094669 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:41.094855 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.094861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.095498 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.095499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.096283 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.096285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.096636 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.096637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.096742 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.097238 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:41.097246 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:41.098541 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:41.100197 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.100204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.100206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.100508 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.100743 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.100743 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.100743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.100842 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.101171 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:41.101176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:41.101290 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.101373 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.101489 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260926 13:07:41.101667 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.093242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:33791 (local address 127.2.86.62:34193)
0926 13:07:41.093306 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.093308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.093535 (+ 227us) server_negotiation.cc:407] Connection header received
0926 13:07:41.093992 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.093993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.094004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.094061 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.094933 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.095422 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.096362 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.096566 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.096616 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.098639 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.098641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.098642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:41.098649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.100104 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.100581 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.100582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.100582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.100673 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.100956 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.100957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.100958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.101099 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.101175 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.101414 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.101525 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 13:07:41.102545 19590 heartbeater.cc:347] Connected to a master server at 127.2.86.62:34193
I20260926 13:07:41.102660 19590 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:41.102851 19590 heartbeater.cc:511] Master 127.2.86.62:34193 requested a full tablet report, sending...
I20260926 13:07:41.103390 19399 ts_manager.cc:194] Registered new tserver with Master: 3da78569b09044dcb932b876bbe85580 (127.2.86.1:33003)
I20260926 13:07:41.104317 19399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:33791
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:07:41.123945 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:41.204485 19595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:41.204685 19595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:41.204708 19595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:41.205902 19595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:41.205945 19595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:41.205998 19595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:41.207433 19595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:41.207680 19595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:41.207895 19595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:41.212714 19603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.212761 19595 server_base.cc:1034] running on GCE node
W20260926 13:07:41.212706 19601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:41.212706 19600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.213204 19595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:41.213438 19595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:41.214592 19595 hybrid_clock.cc:648] HybridClock initialized: now 1790428061214595 us; error 38 us; skew 500 ppm
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:41.217231 19595 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:07:41.217587 19595 webserver.cc:460] Webserver started at http://127.2.86.2:36023/ using document root <none> and password file <none>
I20260926 13:07:41.217763 19595 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:41.217801 19595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:41.217885 19595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:41.218876 19595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "acad43b3ea024144852fdd0d238065f2"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "b01ccd086198e9a9ce91fe9d7e7e3c43"
server_key_iv: "878ae4b460a45f23ce9f949094d315a5"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.219218 19595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "acad43b3ea024144852fdd0d238065f2"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "b01ccd086198e9a9ce91fe9d7e7e3c43"
server_key_iv: "878ae4b460a45f23ce9f949094d315a5"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.220466 19595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 13:07:41.221264 19610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.221436 19595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:07:41.221529 19595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "acad43b3ea024144852fdd0d238065f2"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "b01ccd086198e9a9ce91fe9d7e7e3c43"
server_key_iv: "878ae4b460a45f23ce9f949094d315a5"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.221607 19595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:41.230180 19595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:41.231927 19595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:41.232090 19595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:41.232307 19595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:41.232637 19595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:41.232692 19595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.232731 19595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:41.232759 19595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.238508 19595 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:38951
I20260926 13:07:41.238545 19723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:38951 every 8 connection(s)
I20260926 13:07:41.238857 19595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:41.247659 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.239572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:53499 (local address 127.2.86.62:34193)
0926 13:07:41.239661 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.239665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.239854 (+ 189us) server_negotiation.cc:407] Connection header received
0926 13:07:41.240315 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.240316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.240326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.240373 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.241337 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.241845 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.242795 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.243029 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.243088 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.245081 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.245082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.245082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:41.245089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.246577 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.246919 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.246920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.246920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.246968 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.247168 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.247169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.247169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.247263 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.247312 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.247528 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.247597 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 13:07:41.247843 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.239552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:34193 (local address 127.2.86.2:53499)
0926 13:07:41.239734 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.239769 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.240232 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.240432 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.240451 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.241074 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:41.241259 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.241264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.241922 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.241923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.242693 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.242695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.243104 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.243105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.243209 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.243633 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:41.243640 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:41.245002 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:41.246645 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.246650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.246652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.246853 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.247024 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.247024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.247024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.247115 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.247316 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:41.247319 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:41.247423 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.247511 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.247621 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
I20260926 13:07:41.248440 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19595
I20260926 13:07:41.248473 19724 heartbeater.cc:347] Connected to a master server at 127.2.86.62:34193
I20260926 13:07:41.248560 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:41.248570 19724 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:41.248750 19724 heartbeater.cc:511] Master 127.2.86.62:34193 requested a full tablet report, sending...
I20260926 13:07:41.248798 2392 external_mini_cluster.cc:1428] Setting key 9a36e7224bb2c383e4bbd4b754541669
I20260926 13:07:41.249164 19399 ts_manager.cc:194] Registered new tserver with Master: acad43b3ea024144852fdd0d238065f2 (127.2.86.2:38951)
I20260926 13:07:41.249615 19399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:53499
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:07:41.271446 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:41.352484 19729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:41.352720 19729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:41.352758 19729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:41.353960 19729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:41.354045 19729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:41.354091 19729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:41.355533 19729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:41.355803 19729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:41.356029 19729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:41.360922 19734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.361042 19729 server_base.cc:1034] running on GCE node
W20260926 13:07:41.360947 19735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:41.360965 19737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.361325 19729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:41.361580 19729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:41.362733 19729 hybrid_clock.cc:648] HybridClock initialized: now 1790428061362723 us; error 31 us; skew 500 ppm
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:41.365305 19729 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:07:41.365816 19729 webserver.cc:460] Webserver started at http://127.2.86.3:44309/ using document root <none> and password file <none>
I20260926 13:07:41.366060 19729 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:41.366106 19729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:41.366255 19729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:41.367293 19729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "00b35c38313e426887128a6693608e14"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "938b80afb44934bb49509761159c2201"
server_key_iv: "381ab718757d1432a0c4e5fcda58fd67"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.367654 19729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "00b35c38313e426887128a6693608e14"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "938b80afb44934bb49509761159c2201"
server_key_iv: "381ab718757d1432a0c4e5fcda58fd67"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.369026 19729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:07:41.369764 19744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.369930 19729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:41.370057 19729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "00b35c38313e426887128a6693608e14"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "938b80afb44934bb49509761159c2201"
server_key_iv: "381ab718757d1432a0c4e5fcda58fd67"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.370133 19729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:41.393105 19729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:41.395503 19729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:41.395643 19729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:41.395841 19729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:41.396166 19729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:41.396224 19729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.396258 19729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:41.396299 19729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.402199 19729 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:39083
I20260926 13:07:41.402249 19857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:39083 every 8 connection(s)
I20260926 13:07:41.402575 19729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:41.406191 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19729
I20260926 13:07:41.406315 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:41.406522 2392 external_mini_cluster.cc:1428] Setting key b9a1aa859e631e91637abd4b3fb6082b
Sep 26 13:07:41 dist-test-slave-j4lq krb5kdc[19354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428061, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:41.411890 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.403278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:60867 (local address 127.2.86.62:34193)
0926 13:07:41.403342 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.403344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.403539 (+ 195us) server_negotiation.cc:407] Connection header received
0926 13:07:41.404046 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.404047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.404058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.404114 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.405048 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.405542 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.406837 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.407169 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.407259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.409404 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.409406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.409406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:41.409412 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.410835 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.411176 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.411177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.411177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.411225 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.411391 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.411392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.411392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.411487 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.411537 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.411769 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.411827 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:07:41.412098 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.403255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:34193 (local address 127.2.86.3:60867)
0926 13:07:41.403435 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.403470 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.403933 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.404196 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.404199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.404759 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:41.404972 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.404978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.405617 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.405618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.406777 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.406780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.407311 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.407312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.407453 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.407947 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:41.407955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:41.409327 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:41.410903 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.410908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.410910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.411113 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.411264 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:41.411265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:41.411265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:41.411343 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:41.411544 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:41.411548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:41.411664 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.411742 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.411854 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260926 13:07:41.412578 19858 heartbeater.cc:347] Connected to a master server at 127.2.86.62:34193
I20260926 13:07:41.412655 19858 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:41.412793 19858 heartbeater.cc:511] Master 127.2.86.62:34193 requested a full tablet report, sending...
I20260926 13:07:41.413163 19399 ts_manager.cc:194] Registered new tserver with Master: 00b35c38313e426887128a6693608e14 (127.2.86.3:39083)
I20260926 13:07:41.413480 19399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:60867
I20260926 13:07:41.418344 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:41.425607 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.419752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.2.86.62:34193)
0926 13:07:41.419801 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.419803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.419811 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:07:41.419830 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.419831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.419841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.419891 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.420487 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.421009 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.421688 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.421858 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.421903 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.423225 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.423226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.423227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:41.423233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.424431 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.424739 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.424740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.424740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.424785 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:41.425060 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:41.425060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:41.425060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:41.425188 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.425301 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.425431 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.425506 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 13:07:41.427294 19399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:41.428799 19399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:41.435340 19871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.431359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.2.86.1:33003)
0926 13:07:41.431756 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.431761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.431771 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:07:41.431812 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.431814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.431827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.431900 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:41.432118 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.433207 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.434154 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.434906 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.435098 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.435161 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.435209 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.435254 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":77,"threads_started":1}
I20260926 13:07:41.435340 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.430981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.2.86.2:38951)
0926 13:07:41.431170 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.431176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.431411 (+ 235us) server_negotiation.cc:407] Connection header received
0926 13:07:41.431503 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.431506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.431527 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.431607 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:41.431802 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.432631 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.434151 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.434881 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.434918 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.434985 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.435220 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.435262 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260926 13:07:41.435353 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.430879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:38951 (local address 127.0.0.1:48122)
0926 13:07:41.431334 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.431343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.431420 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.431649 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.431653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.431655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:41.431723 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.431724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.432731 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.432732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.434049 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.434052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.435045 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.435045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.435101 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.435118 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.435191 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.435253 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":32,"thread_start_us":73,"threads_started":1}
I20260926 13:07:41.435503 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.431140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33003 (local address 127.0.0.1:51808)
0926 13:07:41.431500 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.431508 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.431542 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.431972 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.431973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.431976 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:41.432032 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.432034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.433300 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.433301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.434045 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.434047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.435008 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.435009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.435101 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.435118 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.435177 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.435253 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":52,"threads_started":1}
I20260926 13:07:41.435981 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.431052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.2.86.3:39083)
0926 13:07:41.431258 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.431262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.431382 (+ 120us) server_negotiation.cc:407] Connection header received
0926 13:07:41.431506 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.431509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.431525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.431629 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:41.431921 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.433038 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.434264 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.434919 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.434985 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.435038 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.435857 (+ 819us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.435902 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260926 13:07:41.436199 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.430879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:39083 (local address 127.0.0.1:58030)
0926 13:07:41.431248 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.431300 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.431384 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.431803 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.431803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.431805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:41.431847 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.431848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.433135 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.433136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.434190 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.434192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.435288 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.435289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.435347 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.435350 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.435398 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.436134 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":25,"thread_start_us":65,"threads_started":1}
I20260926 13:07:41.436127 19658 tablet_service.cc:1467] Processing CreateTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 (DEFAULT_TABLE table=test-table [id=0659b9d5cb5d43b186f67c5cc44e8b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:41.436396 19524 tablet_service.cc:1467] Processing CreateTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 (DEFAULT_TABLE table=test-table [id=0659b9d5cb5d43b186f67c5cc44e8b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:41.436519 19658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fdaf631a66f426ab44bafee9a0cd323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:41.436743 19524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fdaf631a66f426ab44bafee9a0cd323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:41.436857 19792 tablet_service.cc:1467] Processing CreateTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 (DEFAULT_TABLE table=test-table [id=0659b9d5cb5d43b186f67c5cc44e8b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:41.437209 19792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fdaf631a66f426ab44bafee9a0cd323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:41.439152 19872 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Bootstrap starting.
I20260926 13:07:41.440086 19873 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Bootstrap starting.
I20260926 13:07:41.440135 19872 tablet_bootstrap.cc:654] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:41.440481 19874 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Bootstrap starting.
I20260926 13:07:41.440730 19872 log.cc:824] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:41.440834 19873 tablet_bootstrap.cc:654] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:41.441223 19874 tablet_bootstrap.cc:654] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:41.441452 19873 log.cc:824] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:41.441582 19872 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: No bootstrap required, opened a new log
I20260926 13:07:41.441653 19872 ts_tablet_manager.cc:1397] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:07:41.441787 19874 log.cc:824] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:41.442165 19873 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: No bootstrap required, opened a new log
I20260926 13:07:41.442248 19873 ts_tablet_manager.cc:1397] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 13:07:41.442507 19874 tablet_bootstrap.cc:492] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: No bootstrap required, opened a new log
I20260926 13:07:41.442579 19874 ts_tablet_manager.cc:1397] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:41.443779 19873 raft_consensus.cc:348] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.443794 19872 raft_consensus.cc:348] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.443953 19873 raft_consensus.cc:374] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:41.443969 19872 raft_consensus.cc:374] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:41.444013 19873 raft_consensus.cc:729] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da78569b09044dcb932b876bbe85580, State: Initialized, Role: FOLLOWER
I20260926 13:07:41.444047 19872 raft_consensus.cc:729] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acad43b3ea024144852fdd0d238065f2, State: Initialized, Role: FOLLOWER
I20260926 13:07:41.444170 19872 consensus_queue.cc:260] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.444170 19873 consensus_queue.cc:260] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.444449 19873 ts_tablet_manager.cc:1428] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:07:41.444446 19872 ts_tablet_manager.cc:1428] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:41.444597 19590 heartbeater.cc:503] Master 127.2.86.62:34193 was elected leader, sending a full tablet report...
I20260926 13:07:41.444917 19724 heartbeater.cc:503] Master 127.2.86.62:34193 was elected leader, sending a full tablet report...
I20260926 13:07:41.444993 19874 raft_consensus.cc:348] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.445137 19874 raft_consensus.cc:374] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:41.445199 19874 raft_consensus.cc:729] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b35c38313e426887128a6693608e14, State: Initialized, Role: FOLLOWER
I20260926 13:07:41.445329 19874 consensus_queue.cc:260] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.445611 19874 ts_tablet_manager.cc:1428] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:07:41.445786 19858 heartbeater.cc:503] Master 127.2.86.62:34193 was elected leader, sending a full tablet report...
W20260926 13:07:41.489517 19725 tablet.cc:2369] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:07:41.593318 19591 tablet.cc:2369] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:07:41.635955 19879 raft_consensus.cc:482] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:41.636134 19879 raft_consensus.cc:504] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.636545 19879 leader_election.cc:288] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acad43b3ea024144852fdd0d238065f2 (127.2.86.2:38951), 00b35c38313e426887128a6693608e14 (127.2.86.3:39083)
I20260926 13:07:41.639693 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.636837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:60091 (local address 127.2.86.2:38951)
0926 13:07:41.636904 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.636906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.636998 (+ 92us) server_negotiation.cc:407] Connection header received
0926 13:07:41.637059 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.637060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.637074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.637129 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:41.637302 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.637862 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.638932 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.639332 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.639385 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.639393 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.639582 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.639625 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:07:41.639716 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.636737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:38951 (local address 127.2.86.1:60091)
0926 13:07:41.636899 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.636910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.636998 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.637150 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.637151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.637153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:41.637232 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.637235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.637966 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.637967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.638836 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.638839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.639412 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.639413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.639512 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.639515 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.639558 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.639623 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260926 13:07:41.639703 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.636847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:50605 (local address 127.2.86.3:39083)
0926 13:07:41.636910 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.636913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.637004 (+ 91us) server_negotiation.cc:407] Connection header received
0926 13:07:41.637099 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.637100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.637129 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.637185 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:41.637524 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.638085 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.638991 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.639352 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.639444 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.639454 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.639605 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.639640 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:07:41.639753 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.636760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:39083 (local address 127.2.86.1:50605)
0926 13:07:41.636906 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.636916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.637020 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.637204 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.637205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.637207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:41.637298 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.637300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.638171 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.638171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.638904 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.638906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.639466 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.639467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.639518 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.639521 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.639590 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.639656 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":13,"thread_start_us":70,"threads_started":1}
I20260926 13:07:41.640198 19678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fdaf631a66f426ab44bafee9a0cd323" candidate_uuid: "3da78569b09044dcb932b876bbe85580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad43b3ea024144852fdd0d238065f2" is_pre_election: true
I20260926 13:07:41.640223 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fdaf631a66f426ab44bafee9a0cd323" candidate_uuid: "3da78569b09044dcb932b876bbe85580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b35c38313e426887128a6693608e14" is_pre_election: true
I20260926 13:07:41.640349 19678 raft_consensus.cc:2454] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da78569b09044dcb932b876bbe85580 in term 0.
I20260926 13:07:41.640349 19812 raft_consensus.cc:2454] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da78569b09044dcb932b876bbe85580 in term 0.
I20260926 13:07:41.640594 19477 leader_election.cc:302] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da78569b09044dcb932b876bbe85580, acad43b3ea024144852fdd0d238065f2; no voters:
I20260926 13:07:41.640744 19879 raft_consensus.cc:2781] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:41.640802 19879 raft_consensus.cc:482] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:41.640830 19879 raft_consensus.cc:3037] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:41.641613 19879 raft_consensus.cc:504] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.641752 19879 leader_election.cc:288] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [CANDIDATE]: Term 1 election: Requested vote from peers acad43b3ea024144852fdd0d238065f2 (127.2.86.2:38951), 00b35c38313e426887128a6693608e14 (127.2.86.3:39083)
I20260926 13:07:41.641912 19678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fdaf631a66f426ab44bafee9a0cd323" candidate_uuid: "3da78569b09044dcb932b876bbe85580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad43b3ea024144852fdd0d238065f2"
I20260926 13:07:41.641914 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fdaf631a66f426ab44bafee9a0cd323" candidate_uuid: "3da78569b09044dcb932b876bbe85580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00b35c38313e426887128a6693608e14"
I20260926 13:07:41.642005 19812 raft_consensus.cc:3037] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:41.642010 19678 raft_consensus.cc:3037] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:41.642716 19678 raft_consensus.cc:2454] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da78569b09044dcb932b876bbe85580 in term 1.
I20260926 13:07:41.642822 19812 raft_consensus.cc:2454] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da78569b09044dcb932b876bbe85580 in term 1.
I20260926 13:07:41.642935 19477 leader_election.cc:302] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da78569b09044dcb932b876bbe85580, acad43b3ea024144852fdd0d238065f2; no voters:
I20260926 13:07:41.643036 19879 raft_consensus.cc:2781] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:41.643184 19879 raft_consensus.cc:686] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [term 1 LEADER]: Becoming Leader. State: Replica: 3da78569b09044dcb932b876bbe85580, State: Running, Role: LEADER
I20260926 13:07:41.643374 19879 consensus_queue.cc:237] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } }
I20260926 13:07:41.644352 19399 catalog_manager.cc:5495] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da78569b09044dcb932b876bbe85580 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "3da78569b09044dcb932b876bbe85580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da78569b09044dcb932b876bbe85580" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 } health_report { overall_health: UNKNOWN } } }
W20260926 13:07:41.653040 19859 tablet.cc:2369] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:07:41.661348 19871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.659006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.2.86.1:33003)
0926 13:07:41.659056 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.659058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.659087 (+ 29us) server_negotiation.cc:407] Connection header received
0926 13:07:41.659168 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.659169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.659186 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.659239 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:41.659348 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.659820 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.660591 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.660769 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.660818 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.661072 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:41.661174 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:41.661237 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.661279 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.661295 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:07:41.663161 19678 raft_consensus.cc:1261] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 1 FOLLOWER]: Refusing update from remote peer 3da78569b09044dcb932b876bbe85580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:41.663161 19812 raft_consensus.cc:1261] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 1 FOLLOWER]: Refusing update from remote peer 3da78569b09044dcb932b876bbe85580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:41.663583 19883 consensus_queue.cc:1035] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acad43b3ea024144852fdd0d238065f2" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 38951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:41.663755 19879 consensus_queue.cc:1035] T 4fdaf631a66f426ab44bafee9a0cd323 P 3da78569b09044dcb932b876bbe85580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00b35c38313e426887128a6693608e14" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:41.668398 19399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:41.668509 19399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:41.669263 19399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff1c63bba1d94384bfef4a9d3512272f: Sending DeleteTablet for 3 replicas of tablet 4fdaf631a66f426ab44bafee9a0cd323
I20260926 13:07:41.669565 19658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:41 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:48122
I20260926 13:07:41.669643 19524 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:41 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:51808
I20260926 13:07:41.669669 19792 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fdaf631a66f426ab44bafee9a0cd323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:41 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:58030
I20260926 13:07:41.669886 19896 tablet_replica.cc:330] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: stopping tablet replica
I20260926 13:07:41.669921 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19461
I20260926 13:07:41.670554 19896 raft_consensus.cc:2229] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:41.670711 19896 raft_consensus.cc:2258] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:41.671175 19896 ts_tablet_manager.cc:1905] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:41.673128 19896 ts_tablet_manager.cc:1918] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:41.673221 19896 log.cc:1196] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/4fdaf631a66f426ab44bafee9a0cd323
I20260926 13:07:41.673461 19896 ts_tablet_manager.cc:1939] T 4fdaf631a66f426ab44bafee9a0cd323 P acad43b3ea024144852fdd0d238065f2: Deleting consensus metadata
I20260926 13:07:41.674191 19383 catalog_manager.cc:4841] TS acad43b3ea024144852fdd0d238065f2 (127.2.86.2:38951): tablet 4fdaf631a66f426ab44bafee9a0cd323 (table test-table [id=0659b9d5cb5d43b186f67c5cc44e8b28]) successfully deleted
W20260926 13:07:41.676421 19383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:33003 (error 108)
I20260926 13:07:41.676595 19898 tablet_replica.cc:330] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: stopping tablet replica
I20260926 13:07:41.676725 19898 raft_consensus.cc:2229] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:41.676851 19898 raft_consensus.cc:2258] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:41.676913 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.676749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33003 (local address 127.0.0.1:51818)
0926 13:07:41.676801 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.676834 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:07:41.677062 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19595
W20260926 13:07:41.677095 19383 catalog_manager.cc:4584] TS 3da78569b09044dcb932b876bbe85580 (127.2.86.1:33003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fdaf631a66f426ab44bafee9a0cd323: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33003: connect: Connection refused (error 111)
I20260926 13:07:41.679050 19898 ts_tablet_manager.cc:1905] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:41.680581 19898 ts_tablet_manager.cc:1918] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:41.680668 19898 log.cc:1196] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/4fdaf631a66f426ab44bafee9a0cd323
I20260926 13:07:41.680966 19898 ts_tablet_manager.cc:1939] T 4fdaf631a66f426ab44bafee9a0cd323 P 00b35c38313e426887128a6693608e14: Deleting consensus metadata
I20260926 13:07:41.681381 19384 catalog_manager.cc:4841] TS 00b35c38313e426887128a6693608e14 (127.2.86.3:39083): tablet 4fdaf631a66f426ab44bafee9a0cd323 (table test-table [id=0659b9d5cb5d43b186f67c5cc44e8b28]) successfully deleted
I20260926 13:07:41.681695 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19729
I20260926 13:07:41.686352 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19367
2026-09-26T13:07:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2057 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T13:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:41Z Disabled control of system clock
I20260926 13:07:41.713060 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41933
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:41933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:41.791210 19903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:41.791412 19903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:41.791432 19903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:41.792487 19903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:41.792519 19903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:41.792532 19903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:41.792544 19903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:41.792557 19903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:41.793823 19903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:41933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:41.794005 19903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:41.794236 19903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:41.799144 19908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.799283 19903 server_base.cc:1034] running on GCE node
W20260926 13:07:41.799213 19909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:41.799178 19911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.799639 19903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:41.799925 19903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:41.801085 19903 hybrid_clock.cc:648] HybridClock initialized: now 1790428061801063 us; error 34 us; skew 500 ppm
I20260926 13:07:41.802331 19903 webserver.cc:460] Webserver started at http://127.2.86.62:36289/ using document root <none> and password file <none>
I20260926 13:07:41.802596 19903 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:41.802670 19903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:41.802814 19903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:41.803857 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "83793e8f31e24f75b53837b2f50933a4"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "9098e8840d1ab5c46028dab69d9a39db"
server_key_iv: "dc0dce98ef7a76be9d463addb470b15c"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.804208 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "83793e8f31e24f75b53837b2f50933a4"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "9098e8840d1ab5c46028dab69d9a39db"
server_key_iv: "dc0dce98ef7a76be9d463addb470b15c"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.805563 19903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:41.806218 19917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.806418 19903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:41.806521 19903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "83793e8f31e24f75b53837b2f50933a4"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "9098e8840d1ab5c46028dab69d9a39db"
server_key_iv: "dc0dce98ef7a76be9d463addb470b15c"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.806576 19903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:41.819900 19903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:41.820142 19903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:41.820242 19903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:41.823804 19903 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:41933
I20260926 13:07:41.823853 19969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:41933 every 8 connection(s)
I20260926 13:07:41.824158 19903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:41.825102 19970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:41.826802 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19903
I20260926 13:07:41.826877 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:41.827086 2392 external_mini_cluster.cc:1428] Setting key bab2c2ae27309fee4a02f09cb7b013f1
I20260926 13:07:41.827469 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: Bootstrap starting.
I20260926 13:07:41.828083 19970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:41.828382 19970 log.cc:824] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:41.829133 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: No bootstrap required, opened a new log
I20260926 13:07:41.831281 19970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:07:41.831461 19970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:41.831524 19970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83793e8f31e24f75b53837b2f50933a4, State: Initialized, Role: FOLLOWER
I20260926 13:07:41.831660 19970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:07:41.831755 19970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:41.831799 19970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:41.831882 19970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:41.832515 19970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:07:41.832641 19970 leader_election.cc:302] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83793e8f31e24f75b53837b2f50933a4; no voters:
I20260926 13:07:41.832813 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.827612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.2.86.62:41933)
0926 13:07:41.827802 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.827810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.827833 (+ 23us) server_negotiation.cc:407] Connection header received
0926 13:07:41.828312 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.828319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.828377 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.828530 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.828823 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.829778 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.830922 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.831295 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.831380 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.832092 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.832095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.832095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:41.832259 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.832290 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:41.832300 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.832348 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.832449 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.832503 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260926 13:07:41.832909 19970 leader_election.cc:288] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:41.832931 19975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:41.833043 19975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [term 1 LEADER]: Becoming Leader. State: Replica: 83793e8f31e24f75b53837b2f50933a4, State: Running, Role: LEADER
I20260926 13:07:41.833168 19970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:41.833469 19975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } }
I20260926 13:07:41.833817 19975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83793e8f31e24f75b53837b2f50933a4. Latest consensus state: current_term: 1 leader_uuid: "83793e8f31e24f75b53837b2f50933a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } } }
I20260926 13:07:41.833827 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83793e8f31e24f75b53837b2f50933a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83793e8f31e24f75b53837b2f50933a4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41933 } } }
I20260926 13:07:41.833909 19975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:41.833916 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:41.834103 19983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:41.834564 19983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:41.837899 19983 catalog_manager.cc:1347] Generated new cluster ID: 107c805d40c84e88a21e7afee3f11222
I20260926 13:07:41.837960 19983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:41.853196 19983 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:41.853677 19983 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:41.857197 19983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: Generated new TSK 0
I20260926 13:07:41.857362 19983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:41.864769 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--builtin_ntp_servers=127.2.86.20:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:41.949919 19994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:41.950167 19994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:41.950209 19994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:41.951478 19994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:41.951545 19994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:41.951591 19994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:41.953138 19994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:41.953403 19994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:41.953742 19994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:41.959022 20002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:41.959084 20000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.959103 19994 server_base.cc:1034] running on GCE node
W20260926 13:07:41.959041 19999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:41.959419 19994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:41.959677 19994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:41.960857 19994 hybrid_clock.cc:648] HybridClock initialized: now 1790428061960829 us; error 41 us; skew 500 ppm
I20260926 13:07:41.962167 19994 webserver.cc:460] Webserver started at http://127.2.86.1:33005/ using document root <none> and password file <none>
I20260926 13:07:41.962410 19994 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:41.962481 19994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:41.962638 19994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:41.963706 19994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "cb59262332be4ceb80c0452da97b8023"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "4622f58deaa079c8c6adda5012abdb6b"
server_key_iv: "81200f0043b535f8a4c523f8983a1876"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.964061 19994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "cb59262332be4ceb80c0452da97b8023"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "4622f58deaa079c8c6adda5012abdb6b"
server_key_iv: "81200f0043b535f8a4c523f8983a1876"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.965404 19994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:41.966104 20008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.966264 19994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:41.966359 19994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "cb59262332be4ceb80c0452da97b8023"
format_stamp: "Formatted at 2026-09-26 13:07:41 on dist-test-slave-j4lq"
server_key: "4622f58deaa079c8c6adda5012abdb6b"
server_key_iv: "81200f0043b535f8a4c523f8983a1876"
server_key_version: "encryptionkey@0"
I20260926 13:07:41.966439 19994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:41.981755 19994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:41.982056 19994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:41.982226 19994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:41.982476 19994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:41.982788 19994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:41.982841 19994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.982882 19994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:41.982908 19994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:41.988341 19994 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:40655
I20260926 13:07:41.988411 20121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:40655 every 8 connection(s)
I20260926 13:07:41.988734 19994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:07:41.989688 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 19994
I20260926 13:07:41.989789 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:41.989993 2392 external_mini_cluster.cc:1428] Setting key 6c08dfa7c08a53e2ec87f07a3881f141
I20260926 13:07:41.990927 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--builtin_ntp_servers=127.2.86.20:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:07:41.994262 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.989447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:48595 (local address 127.2.86.62:41933)
0926 13:07:41.989487 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:41.989490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:41.989794 (+ 304us) server_negotiation.cc:407] Connection header received
0926 13:07:41.990443 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:41.990444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:41.990456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:41.990502 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:41.991305 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.991804 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.992793 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.993055 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.993129 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.993992 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:41.993993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:41.993994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:41.993998 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:41.994012 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:41.994016 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:41.994085 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:41.994156 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:07:41.994186 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:07:41.994475 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:41.989456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:41933 (local address 127.2.86.1:48595)
0926 13:07:41.989674 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:41.989707 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:41.990332 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:41.990571 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:41.990574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:41.990584 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:41.991209 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.991215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.991899 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.991900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.992689 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:41.992691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:41.993149 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:41.993150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:41.993290 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:41.993848 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:41.993848 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:41.993875 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:41.993896 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:41.994085 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:41.994091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:41.994094 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:41.994160 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:07:41.994260 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260926 13:07:41.995016 20122 heartbeater.cc:347] Connected to a master server at 127.2.86.62:41933
I20260926 13:07:41.995115 20122 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:41.995293 20122 heartbeater.cc:511] Master 127.2.86.62:41933 requested a full tablet report, sending...
I20260926 13:07:41.995805 19934 ts_manager.cc:194] Registered new tserver with Master: cb59262332be4ceb80c0452da97b8023 (127.2.86.1:40655)
I20260926 13:07:41.996423 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.1:48595
W20260926 13:07:42.070528 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:42.070746 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:42.070794 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:42.071919 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:42.071978 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:42.072026 20125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:42.073413 20125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:42.073658 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:42.073927 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:42.078810 20130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:42.078850 20131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:42.078984 20125 server_base.cc:1034] running on GCE node
W20260926 13:07:42.079015 20133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:42.079249 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:42.079532 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:42.080698 20125 hybrid_clock.cc:648] HybridClock initialized: now 1790428062080677 us; error 36 us; skew 500 ppm
I20260926 13:07:42.082012 20125 webserver.cc:460] Webserver started at http://127.2.86.2:33099/ using document root <none> and password file <none>
I20260926 13:07:42.082248 20125 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:42.082316 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:42.082444 20125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:42.083451 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "647b87c3dba64efd8c047639d64e5ae6"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "49a7eafc575220442b6d47ce54b5cf09"
server_key_iv: "e533c21c751c8a566b7bf9bddb6a22a0"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.083798 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "647b87c3dba64efd8c047639d64e5ae6"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "49a7eafc575220442b6d47ce54b5cf09"
server_key_iv: "e533c21c751c8a566b7bf9bddb6a22a0"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.085031 20125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:42.085733 20139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.085927 20125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:42.086024 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "647b87c3dba64efd8c047639d64e5ae6"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "49a7eafc575220442b6d47ce54b5cf09"
server_key_iv: "e533c21c751c8a566b7bf9bddb6a22a0"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.086103 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:42.094920 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:42.095187 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:42.095328 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:42.095528 20125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:42.095822 20125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:42.095876 20125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.095916 20125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:42.095942 20125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.101511 20125 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:42941
I20260926 13:07:42.101567 20252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:42941 every 8 connection(s)
I20260926 13:07:42.101853 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:42.105695 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20125
I20260926 13:07:42.105796 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:42.106016 2392 external_mini_cluster.cc:1428] Setting key 638dc0d67d780a6e01476de47e9fe523
I20260926 13:07:42.106874 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.102503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:34575 (local address 127.2.86.62:41933)
0926 13:07:42.102573 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.102576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.102806 (+ 230us) server_negotiation.cc:407] Connection header received
0926 13:07:42.103245 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.103246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.103256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.103304 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:42.103913 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.104399 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.105415 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.105608 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.105656 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.106575 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:42.106577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:42.106577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:42.106581 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:42.106596 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:42.106599 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:42.106652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.106752 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.106778 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:07:42.107082 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.102490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:41933 (local address 127.2.86.2:34575)
0926 13:07:42.102693 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.102724 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.103167 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.103362 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.103364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.103372 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:42.103851 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.103856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.104476 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.104476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.105339 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.105341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.105687 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.105688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.105853 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.106395 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:42.106396 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:42.106456 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:42.106483 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:42.106662 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:42.106666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:42.106669 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.106739 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.106855 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260926 13:07:42.107179 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--builtin_ntp_servers=127.2.86.20:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:07:42.107559 20253 heartbeater.cc:347] Connected to a master server at 127.2.86.62:41933
I20260926 13:07:42.107671 20253 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:42.107841 20253 heartbeater.cc:511] Master 127.2.86.62:41933 requested a full tablet report, sending...
I20260926 13:07:42.108177 19934 ts_manager.cc:194] Registered new tserver with Master: 647b87c3dba64efd8c047639d64e5ae6 (127.2.86.2:42941)
I20260926 13:07:42.108604 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.2:34575
W20260926 13:07:42.185105 20256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:42.185294 20256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:42.185315 20256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:42.186434 20256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:42.186472 20256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:42.186497 20256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:42.187837 20256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:42.188102 20256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:42.188448 20256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:42.193686 20264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:42.193674 20261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:42.193704 20262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:42.194129 20256 server_base.cc:1034] running on GCE node
I20260926 13:07:42.194336 20256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:42.194602 20256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:42.195786 20256 hybrid_clock.cc:648] HybridClock initialized: now 1790428062195777 us; error 27 us; skew 500 ppm
I20260926 13:07:42.197024 20256 webserver.cc:460] Webserver started at http://127.2.86.3:43157/ using document root <none> and password file <none>
I20260926 13:07:42.197278 20256 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:42.197322 20256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:42.197463 20256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:42.198488 20256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "63ee7d98230641c39e5bb2b0e3223d1e"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "713ecd9635f40007a37b4cf9acc5cf60"
server_key_iv: "f3468ea3913dd3898009b6f14b33edba"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.198843 20256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "63ee7d98230641c39e5bb2b0e3223d1e"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "713ecd9635f40007a37b4cf9acc5cf60"
server_key_iv: "f3468ea3913dd3898009b6f14b33edba"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.200198 20256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:42.200915 20270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.201110 20256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:42.201208 20256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "63ee7d98230641c39e5bb2b0e3223d1e"
format_stamp: "Formatted at 2026-09-26 13:07:42 on dist-test-slave-j4lq"
server_key: "713ecd9635f40007a37b4cf9acc5cf60"
server_key_iv: "f3468ea3913dd3898009b6f14b33edba"
server_key_version: "encryptionkey@0"
I20260926 13:07:42.201294 20256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:42.208264 20256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:42.208571 20256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:42.208724 20256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:42.208984 20256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:42.209288 20256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:42.209340 20256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.209379 20256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:42.209407 20256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:42.215077 20256 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:46635
I20260926 13:07:42.215122 20383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:46635 every 8 connection(s)
I20260926 13:07:42.215416 20256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:42.220019 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.216056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:56219 (local address 127.2.86.62:41933)
0926 13:07:42.216097 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.216099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.216351 (+ 252us) server_negotiation.cc:407] Connection header received
0926 13:07:42.216784 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.216785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.216795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.216844 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:42.217510 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.217978 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.218897 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.219086 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.219140 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.219809 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:42.219810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:42.219811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:42.219814 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:42.219827 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:42.219830 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:42.219870 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.219929 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.219948 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:07:42.220175 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.216072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:41933 (local address 127.2.86.3:56219)
0926 13:07:42.216232 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.216262 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.216706 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.216959 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.216962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.216970 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:42.217437 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.217441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.218058 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.218059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.218810 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.218812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.219164 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.219164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.219262 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.219693 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:42.219693 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:42.219718 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:42.219738 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:42.219870 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:42.219872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:42.219874 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.219917 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.219986 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260926 13:07:42.220636 20384 heartbeater.cc:347] Connected to a master server at 127.2.86.62:41933
I20260926 13:07:42.220738 20384 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:42.220969 20384 heartbeater.cc:511] Master 127.2.86.62:41933 requested a full tablet report, sending...
I20260926 13:07:42.221329 19934 ts_manager.cc:194] Registered new tserver with Master: 63ee7d98230641c39e5bb2b0e3223d1e (127.2.86.3:46635)
I20260926 13:07:42.221653 19934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.3:56219
I20260926 13:07:42.222041 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20256
I20260926 13:07:42.222114 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:42.222321 2392 external_mini_cluster.cc:1428] Setting key 5b14e7bc1fde2a2d895166d386efe54a
I20260926 13:07:42.223055 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:42.226727 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.224298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.2.86.62:41933)
0926 13:07:42.224332 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.224334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.224434 (+ 100us) server_negotiation.cc:407] Connection header received
0926 13:07:42.224479 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.224479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.224487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.224532 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:42.224651 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.225157 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.225845 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.226010 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.226057 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.226560 (+ 503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:42.226562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:42.226562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:42.226565 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:42.226577 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:42.226580 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:42.226625 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.226660 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.226677 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:07:42.228255 19934 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:42.229795 19934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:42.236147 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.231789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:42941 (local address 127.0.0.1:42240)
0926 13:07:42.232108 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.232117 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.232156 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.232395 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.232399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.232401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:42.232463 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.232465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233879 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.233880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.235184 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.235185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.235844 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.235919 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.235932 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.236008 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.236056 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":111,"thread_start_us":94,"threads_started":1}
I20260926 13:07:42.236269 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.231789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:46635 (local address 127.0.0.1:60958)
0926 13:07:42.232206 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.232214 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.232258 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.232406 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.232407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.232408 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:42.232463 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.232465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233811 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.233812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.235044 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.235062 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.236042 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.236043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.236102 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.236107 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.236158 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.236195 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":33,"thread_start_us":89,"threads_started":1}
I20260926 13:07:42.236368 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.231940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.2.86.1:40655)
0926 13:07:42.232507 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.232511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.232521 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:07:42.232558 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.232561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.232575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.232706 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:42.232917 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233808 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235261 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.235944 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.236008 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.236081 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.236135 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.236183 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":78,"threads_started":1}
I20260926 13:07:42.236399 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.232008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.2.86.2:42941)
0926 13:07:42.232180 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.232183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.232193 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:07:42.232232 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.232236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.232272 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.232364 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:42.232556 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233753 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235274 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.235754 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.235932 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.236277 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.236318 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260926 13:07:42.236554 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.231998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.2.86.3:46635)
0926 13:07:42.232155 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.232161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.232275 (+ 114us) server_negotiation.cc:407] Connection header received
0926 13:07:42.232314 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.232316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.232330 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.232400 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:42.232555 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233735 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235139 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.235852 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.235922 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.236006 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.236424 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.236468 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260926 13:07:42.236799 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.231770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40655 (local address 127.0.0.1:51676)
0926 13:07:42.231989 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.232032 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.232098 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.232777 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.232778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.232779 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:42.232821 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.232822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.233879 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.233880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.235177 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.235180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.236008 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.236008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.236062 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.236065 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.236119 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.236148 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":74,"thread_start_us":55,"threads_started":1}
I20260926 13:07:42.237098 20187 tablet_service.cc:1467] Processing CreateTablet for tablet ccea1742357142bc9aa862cce8a79876 (DEFAULT_TABLE table=test-table [id=65a8e3a4763147548e3e692b7b16797e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:42.237241 20318 tablet_service.cc:1467] Processing CreateTablet for tablet ccea1742357142bc9aa862cce8a79876 (DEFAULT_TABLE table=test-table [id=65a8e3a4763147548e3e692b7b16797e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:42.237479 20318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccea1742357142bc9aa862cce8a79876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:42.237469 20187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccea1742357142bc9aa862cce8a79876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:42.237493 20056 tablet_service.cc:1467] Processing CreateTablet for tablet ccea1742357142bc9aa862cce8a79876 (DEFAULT_TABLE table=test-table [id=65a8e3a4763147548e3e692b7b16797e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:42.237787 20056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccea1742357142bc9aa862cce8a79876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:42.239879 20398 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Bootstrap starting.
I20260926 13:07:42.240517 20398 tablet_bootstrap.cc:654] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:42.240743 20399 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: Bootstrap starting.
I20260926 13:07:42.241009 20398 log.cc:824] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:42.241494 20399 tablet_bootstrap.cc:654] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:42.241787 20398 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: No bootstrap required, opened a new log
I20260926 13:07:42.241802 20399 log.cc:824] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:42.241859 20398 ts_tablet_manager.cc:1397] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:42.242126 20400 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: Bootstrap starting.
I20260926 13:07:42.242434 20399 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: No bootstrap required, opened a new log
I20260926 13:07:42.242511 20399 ts_tablet_manager.cc:1397] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:42.242790 20400 tablet_bootstrap.cc:654] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:42.243113 20400 log.cc:824] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:42.243808 20400 tablet_bootstrap.cc:492] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: No bootstrap required, opened a new log
I20260926 13:07:42.243922 20400 ts_tablet_manager.cc:1397] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:42.244446 20398 raft_consensus.cc:348] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.244603 20398 raft_consensus.cc:374] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:42.244655 20398 raft_consensus.cc:729] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ee7d98230641c39e5bb2b0e3223d1e, State: Initialized, Role: FOLLOWER
I20260926 13:07:42.244779 20398 consensus_queue.cc:260] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.245105 20398 ts_tablet_manager.cc:1428] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 13:07:42.245105 20399 raft_consensus.cc:348] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.245224 20384 heartbeater.cc:503] Master 127.2.86.62:41933 was elected leader, sending a full tablet report...
I20260926 13:07:42.245260 20399 raft_consensus.cc:374] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:42.245301 20399 raft_consensus.cc:729] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647b87c3dba64efd8c047639d64e5ae6, State: Initialized, Role: FOLLOWER
I20260926 13:07:42.245412 20399 consensus_queue.cc:260] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.245656 20399 ts_tablet_manager.cc:1428] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 13:07:42.245708 20253 heartbeater.cc:503] Master 127.2.86.62:41933 was elected leader, sending a full tablet report...
I20260926 13:07:42.246515 20400 raft_consensus.cc:348] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.246622 20400 raft_consensus.cc:374] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:42.246644 20400 raft_consensus.cc:729] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb59262332be4ceb80c0452da97b8023, State: Initialized, Role: FOLLOWER
I20260926 13:07:42.246739 20400 consensus_queue.cc:260] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.246929 20400 ts_tablet_manager.cc:1428] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:42.247045 20122 heartbeater.cc:503] Master 127.2.86.62:41933 was elected leader, sending a full tablet report...
I20260926 13:07:42.257781 20406 raft_consensus.cc:482] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:42.257884 20406 raft_consensus.cc:504] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.258179 20406 leader_election.cc:288] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63ee7d98230641c39e5bb2b0e3223d1e (127.2.86.3:46635), 647b87c3dba64efd8c047639d64e5ae6 (127.2.86.2:42941)
I20260926 13:07:42.261830 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.258407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:40165 (local address 127.2.86.3:46635)
0926 13:07:42.258449 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.258451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.258460 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:42.258487 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.258488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.258497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.258585 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:42.258752 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.259496 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.260986 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.261502 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.261574 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.261583 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.261733 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.261768 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:07:42.261830 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.258407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:45467 (local address 127.2.86.2:42941)
0926 13:07:42.258467 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.258469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.258553 (+ 84us) server_negotiation.cc:407] Connection header received
0926 13:07:42.258615 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.258615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.258623 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.258672 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:42.258812 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.259503 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.260938 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.261500 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.261574 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.261583 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.261733 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.261768 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:07:42.261850 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.258328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:46635 (local address 127.2.86.1:40165)
0926 13:07:42.258385 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.258397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.258433 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.258598 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.258599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.258600 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:42.258686 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.258688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.259576 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.259577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.260837 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.260839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.261584 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.261585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.261657 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.261661 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.261717 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.261763 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":35}
I20260926 13:07:42.261857 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.258338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:42941 (local address 127.2.86.1:45467)
0926 13:07:42.258490 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.258498 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:42.258543 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:42.258688 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:42.258689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:42.258691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:42.258748 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.258749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.259576 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.259577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.260837 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:42.260839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.261596 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.261597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:42.261657 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.261661 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:42.261718 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 13:07:42.261763 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":63,"thread_start_us":82,"threads_started":1}
I20260926 13:07:42.262163 20338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccea1742357142bc9aa862cce8a79876" candidate_uuid: "cb59262332be4ceb80c0452da97b8023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" is_pre_election: true
I20260926 13:07:42.262221 20207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccea1742357142bc9aa862cce8a79876" candidate_uuid: "cb59262332be4ceb80c0452da97b8023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647b87c3dba64efd8c047639d64e5ae6" is_pre_election: true
I20260926 13:07:42.262300 20338 raft_consensus.cc:2454] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb59262332be4ceb80c0452da97b8023 in term 0.
I20260926 13:07:42.262321 20207 raft_consensus.cc:2454] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb59262332be4ceb80c0452da97b8023 in term 0.
I20260926 13:07:42.262516 20011 leader_election.cc:302] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647b87c3dba64efd8c047639d64e5ae6, cb59262332be4ceb80c0452da97b8023; no voters:
I20260926 13:07:42.262667 20406 raft_consensus.cc:2781] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:42.262741 20406 raft_consensus.cc:482] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:42.262782 20406 raft_consensus.cc:3037] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:42.263463 20406 raft_consensus.cc:504] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.263613 20406 leader_election.cc:288] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [CANDIDATE]: Term 1 election: Requested vote from peers 63ee7d98230641c39e5bb2b0e3223d1e (127.2.86.3:46635), 647b87c3dba64efd8c047639d64e5ae6 (127.2.86.2:42941)
I20260926 13:07:42.263746 20338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccea1742357142bc9aa862cce8a79876" candidate_uuid: "cb59262332be4ceb80c0452da97b8023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63ee7d98230641c39e5bb2b0e3223d1e"
I20260926 13:07:42.263749 20207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccea1742357142bc9aa862cce8a79876" candidate_uuid: "cb59262332be4ceb80c0452da97b8023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647b87c3dba64efd8c047639d64e5ae6"
I20260926 13:07:42.263813 20338 raft_consensus.cc:3037] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:42.263844 20207 raft_consensus.cc:3037] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:42.264472 20338 raft_consensus.cc:2454] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb59262332be4ceb80c0452da97b8023 in term 1.
I20260926 13:07:42.264590 20207 raft_consensus.cc:2454] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb59262332be4ceb80c0452da97b8023 in term 1.
I20260926 13:07:42.264642 20012 leader_election.cc:302] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63ee7d98230641c39e5bb2b0e3223d1e, cb59262332be4ceb80c0452da97b8023; no voters:
I20260926 13:07:42.264710 20406 raft_consensus.cc:2781] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:42.264849 20406 raft_consensus.cc:686] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [term 1 LEADER]: Becoming Leader. State: Replica: cb59262332be4ceb80c0452da97b8023, State: Running, Role: LEADER
I20260926 13:07:42.264979 20406 consensus_queue.cc:237] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } }
I20260926 13:07:42.265714 19933 catalog_manager.cc:5495] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb59262332be4ceb80c0452da97b8023 (127.2.86.1). New cstate: current_term: 1 leader_uuid: "cb59262332be4ceb80c0452da97b8023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59262332be4ceb80c0452da97b8023" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 40655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 } health_report { overall_health: UNKNOWN } } }
I20260926 13:07:42.274674 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.272366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.2.86.1:40655)
0926 13:07:42.272431 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:42.272433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:42.272442 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:42.272506 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:42.272507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:42.272522 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:42.272591 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:42.272693 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.273184 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.273947 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:42.274130 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:42.274204 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:42.274358 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:42.274444 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:42.274533 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:42.274552 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 13:07:42.274570 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 13:07:42.276440 20207 raft_consensus.cc:1261] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 1 FOLLOWER]: Refusing update from remote peer cb59262332be4ceb80c0452da97b8023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:42.276440 20338 raft_consensus.cc:1261] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 1 FOLLOWER]: Refusing update from remote peer cb59262332be4ceb80c0452da97b8023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:42.276777 20408 consensus_queue.cc:1035] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63ee7d98230641c39e5bb2b0e3223d1e" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 46635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:42.276939 20406 consensus_queue.cc:1035] T ccea1742357142bc9aa862cce8a79876 P cb59262332be4ceb80c0452da97b8023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647b87c3dba64efd8c047639d64e5ae6" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:42.279306 20411 mvcc.cc:205] Tried to move back new op lower bound from 7333593343081717760 to 7333593343037923328. Current Snapshot: MvccSnapshot[applied={T|T < 7333593343081717760 or (T in {7333593343081717760})}]
I20260926 13:07:42.279639 20412 mvcc.cc:205] Tried to move back new op lower bound from 7333593343081717760 to 7333593343037923328. Current Snapshot: MvccSnapshot[applied={T|T < 7333593343081717760}]
I20260926 13:07:42.280947 19934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:42.281054 19934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:42.281703 19934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83793e8f31e24f75b53837b2f50933a4: Sending DeleteTablet for 3 replicas of tablet ccea1742357142bc9aa862cce8a79876
I20260926 13:07:42.282007 20056 tablet_service.cc:1514] Processing DeleteTablet for tablet ccea1742357142bc9aa862cce8a79876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:42 UTC) from {username='slave'} at 127.0.0.1:51676
I20260926 13:07:42.282027 20318 tablet_service.cc:1514] Processing DeleteTablet for tablet ccea1742357142bc9aa862cce8a79876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:42 UTC) from {username='slave'} at 127.0.0.1:60958
I20260926 13:07:42.282032 20187 tablet_service.cc:1514] Processing DeleteTablet for tablet ccea1742357142bc9aa862cce8a79876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:42 UTC) from {username='slave'} at 127.0.0.1:42240
I20260926 13:07:42.282261 20423 tablet_replica.cc:330] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6: stopping tablet replica
I20260926 13:07:42.282281 20422 tablet_replica.cc:330] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: stopping tablet replica
I20260926 13:07:42.282358 20422 raft_consensus.cc:2229] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:42.282460 20422 raft_consensus.cc:2258] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:42.282524 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19994
I20260926 13:07:42.283268 20422 ts_tablet_manager.cc:1905] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:42.284924 20422 ts_tablet_manager.cc:1918] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:42.285005 20422 log.cc:1196] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/ccea1742357142bc9aa862cce8a79876
I20260926 13:07:42.285281 20422 ts_tablet_manager.cc:1939] T ccea1742357142bc9aa862cce8a79876 P 63ee7d98230641c39e5bb2b0e3223d1e: Deleting consensus metadata
I20260926 13:07:42.285661 19921 catalog_manager.cc:4841] TS 63ee7d98230641c39e5bb2b0e3223d1e (127.2.86.3:46635): tablet ccea1742357142bc9aa862cce8a79876 (table test-table [id=65a8e3a4763147548e3e692b7b16797e]) successfully deleted
W20260926 13:07:42.288561 19918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:40655 (error 108)
I20260926 13:07:42.288610 20423 raft_consensus.cc:2229] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:42.288775 20423 raft_consensus.cc:2258] T ccea1742357142bc9aa862cce8a79876 P 647b87c3dba64efd8c047639d64e5ae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:42.288884 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20125
I20260926 13:07:42.289453 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.289018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:40655 (local address 127.0.0.1:51682)
0926 13:07:42.289379 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.289417 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:40655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341}
W20260926 13:07:42.289733 19918 catalog_manager.cc:4584] TS cb59262332be4ceb80c0452da97b8023 (127.2.86.1:40655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ccea1742357142bc9aa862cce8a79876: Network error: Client connection negotiation failed: client connection to 127.2.86.1:40655: connect: Connection refused (error 111)
I20260926 13:07:42.293629 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20256
I20260926 13:07:42.293712 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:42.293607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:42941 (local address 127.0.0.1:42254)
0926 13:07:42.293652 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:42.293682 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.2:42941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 13:07:42.297850 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 19903
2026-09-26T13:07:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (610 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:42 dist-test-slave-j4lq krb5kdc[20427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:42 dist-test-slave-j4lq krb5kdc[20427](info): set up 2 sockets
Sep 26 13:07:42 dist-test-slave-j4lq krb5kdc[20427](info): commencing operation
krb5kdc: starting...
W20260926 13:07:43.320585 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:07:43.431783 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:37539
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:43.510711 20440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:43.510932 20440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:43.510968 20440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:43.512107 20440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:43.512189 20440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:43.512214 20440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:43.512247 20440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:43.512270 20440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:43.513761 20440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:37539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:43.514038 20440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:43.514256 20440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:43.518895 20448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.518903 20445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.518930 20446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:43.518996 20440 server_base.cc:1034] running on GCE node
I20260926 13:07:43.519367 20440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:43.519675 20440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:43.520849 20440 hybrid_clock.cc:648] HybridClock initialized: now 1790428063520837 us; error 27 us; skew 500 ppm
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:43.523403 20440 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:07:43.523821 20440 webserver.cc:460] Webserver started at http://127.2.86.62:43389/ using document root <none> and password file <none>
I20260926 13:07:43.524061 20440 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:43.524128 20440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:43.524271 20440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:43.525341 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "5689d69157e84b8c870a48158ae29a93"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "de9abf3625c5c8d491f064b570ed7b63"
server_key_iv: "61e14f2d9496e2a049a9934549e50168"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.525697 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "5689d69157e84b8c870a48158ae29a93"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "de9abf3625c5c8d491f064b570ed7b63"
server_key_iv: "61e14f2d9496e2a049a9934549e50168"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.527048 20440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 13:07:43.527742 20455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.527911 20440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:43.528045 20440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "5689d69157e84b8c870a48158ae29a93"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "de9abf3625c5c8d491f064b570ed7b63"
server_key_iv: "61e14f2d9496e2a049a9934549e50168"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.528118 20440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:43.545639 20440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:43.547429 20440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:43.547585 20440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:43.551640 20440 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:37539
I20260926 13:07:43.551676 20507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:37539 every 8 connection(s)
I20260926 13:07:43.552044 20440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:43.552847 20508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:43.555027 20508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: Bootstrap starting.
I20260926 13:07:43.555526 20508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:43.555760 20508 log.cc:824] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:43.556301 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20440
I20260926 13:07:43.556423 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:43.556469 20508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: No bootstrap required, opened a new log
I20260926 13:07:43.556641 2392 external_mini_cluster.cc:1428] Setting key f4b0951c0fefe2febbda4e9f5ac75149
I20260926 13:07:43.558712 20508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } }
I20260926 13:07:43.558882 20508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:43.558915 20508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5689d69157e84b8c870a48158ae29a93, State: Initialized, Role: FOLLOWER
I20260926 13:07:43.559051 20508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } }
I20260926 13:07:43.559136 20508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:43.559176 20508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:43.559221 20508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:43.559862 20508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } }
I20260926 13:07:43.559964 20508 leader_election.cc:302] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5689d69157e84b8c870a48158ae29a93; no voters:
I20260926 13:07:43.560102 20508 leader_election.cc:288] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:43.560166 20513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:43.560313 20508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:43.560351 20513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [term 1 LEADER]: Becoming Leader. State: Replica: 5689d69157e84b8c870a48158ae29a93, State: Running, Role: LEADER
I20260926 13:07:43.560511 20513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } }
I20260926 13:07:43.560881 20514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5689d69157e84b8c870a48158ae29a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } } }
I20260926 13:07:43.560982 20514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:43.560886 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5689d69157e84b8c870a48158ae29a93. Latest consensus state: current_term: 1 leader_uuid: "5689d69157e84b8c870a48158ae29a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5689d69157e84b8c870a48158ae29a93" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 37539 } } }
I20260926 13:07:43.561146 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:43.561159 20520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:43.561616 20520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:43.565924 20520 catalog_manager.cc:1347] Generated new cluster ID: e3fee6d37eb74363b36e9cb923234251
I20260926 13:07:43.565985 20520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:43.568082 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:43.557176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.2.86.62:37539)
0926 13:07:43.557343 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:43.557353 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:43.557387 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:07:43.557941 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:43.557950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:43.558011 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:43.558165 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:43.558724 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.560409 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.561660 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.562183 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.562302 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:43.564721 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:43.564725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:43.564725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:43.564737 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:43.566610 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.566974 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.566978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.566978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.567068 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.567273 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.567274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.567275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.567498 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:43.567675 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:43.567741 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 13:07:43.567870 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260926 13:07:43.575088 20520 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:43.575593 20520 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:43.579098 20520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: Generated new TSK 0
I20260926 13:07:43.579258 20520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:07:43.611909 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:43.712036 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:43.712276 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:43.712316 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:43.713498 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:43.713564 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:43.713609 20534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:43.715065 20534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:43.715327 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:43.715554 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:43.721215 20540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.721256 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.721213 20542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:43.721468 20534 server_base.cc:1034] running on GCE node
I20260926 13:07:43.721685 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:43.721966 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:43.723119 20534 hybrid_clock.cc:648] HybridClock initialized: now 1790428063723097 us; error 38 us; skew 500 ppm
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:43.725664 20534 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:07:43.726123 20534 webserver.cc:460] Webserver started at http://127.2.86.1:46223/ using document root <none> and password file <none>
I20260926 13:07:43.726341 20534 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:43.726385 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:43.726531 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:43.727557 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "2bcd245cf66546118c3afda0c491fe7c"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "57749f8f175c2e9e42c47efcbdee3730"
server_key_iv: "7f38e2e0ba75a949ff867ff8eab2619a"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.727909 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "2bcd245cf66546118c3afda0c491fe7c"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "57749f8f175c2e9e42c47efcbdee3730"
server_key_iv: "7f38e2e0ba75a949ff867ff8eab2619a"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.729267 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:43.730090 20549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.730283 20534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:07:43.730371 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "2bcd245cf66546118c3afda0c491fe7c"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "57749f8f175c2e9e42c47efcbdee3730"
server_key_iv: "7f38e2e0ba75a949ff867ff8eab2619a"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.730443 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:43.751621 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:43.753969 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:43.754134 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:43.754374 20534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:43.754683 20534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:43.754715 20534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.754738 20534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:43.754786 20534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.760813 20534 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:37295
I20260926 13:07:43.760887 20662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:37295 every 8 connection(s)
I20260926 13:07:43.761257 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:43.767493 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20534
I20260926 13:07:43.767632 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:43.767875 2392 external_mini_cluster.cc:1428] Setting key 7d5eb5a53d7604b468ee54d697c41d1a
I20260926 13:07:43.770576 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:43.762037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:42747 (local address 127.2.86.62:37539)
0926 13:07:43.762108 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:43.762110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:43.762358 (+ 248us) server_negotiation.cc:407] Connection header received
0926 13:07:43.762832 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:43.762833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:43.762844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:43.762897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:43.763789 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.764287 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.765254 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.765447 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.765499 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:43.767603 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:43.767605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:43.767606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:43.767614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:43.769155 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.769629 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.769629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.769630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.769688 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.769975 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.769975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.769976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.770070 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:43.770119 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:43.770448 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 13:07:43.770514 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 13:07:43.770867 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:43.762040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:37539 (local address 127.2.86.1:42747)
0926 13:07:43.762175 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:43.762249 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:43.762748 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:43.762951 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:43.762954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:43.763514 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:43.763713 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:43.763718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.764363 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.764364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:43.765186 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:43.765188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.765518 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.765518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:43.765626 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:43.766089 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:43.766097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:43.767494 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:43.769227 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:43.769233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:43.769235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:43.769550 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:43.769755 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:43.769756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:43.769757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:43.769908 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:43.770140 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:43.770147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:43.770314 (+ 167us) client_negotiation.cc:769] Sending connection context
0926 13:07:43.770433 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 13:07:43.770615 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260926 13:07:43.771370 20663 heartbeater.cc:347] Connected to a master server at 127.2.86.62:37539
I20260926 13:07:43.771512 20663 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:43.771692 20663 heartbeater.cc:511] Master 127.2.86.62:37539 requested a full tablet report, sending...
I20260926 13:07:43.772207 20472 ts_manager.cc:194] Registered new tserver with Master: 2bcd245cf66546118c3afda0c491fe7c (127.2.86.1:37295)
I20260926 13:07:43.772950 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:42747
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:07:43.793404 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:43.872262 20668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:43.872483 20668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:43.872519 20668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:43.874068 20668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:43.874133 20668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:43.874190 20668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:43.876773 20668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:43.877067 20668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:43.877345 20668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:43.883486 20674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.883486 20673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:43.883530 20676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:43.884254 20668 server_base.cc:1034] running on GCE node
I20260926 13:07:43.884411 20668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:43.884600 20668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:43.885730 20668 hybrid_clock.cc:648] HybridClock initialized: now 1790428063885720 us; error 28 us; skew 500 ppm
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:43.892707 20668 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:07:43.893244 20668 webserver.cc:460] Webserver started at http://127.2.86.2:36357/ using document root <none> and password file <none>
I20260926 13:07:43.893472 20668 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:43.893539 20668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:43.893692 20668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:43.894721 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "ef3e898e75d74134ac7dea6f330b6399"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "f6a77b9b0b93af83e3ceb963b512989e"
server_key_iv: "e0d056bb34100ff9defe8c4991c45d1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.895080 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "ef3e898e75d74134ac7dea6f330b6399"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "f6a77b9b0b93af83e3ceb963b512989e"
server_key_iv: "e0d056bb34100ff9defe8c4991c45d1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.896365 20668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:43.897181 20683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.897356 20668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:43.897450 20668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "ef3e898e75d74134ac7dea6f330b6399"
format_stamp: "Formatted at 2026-09-26 13:07:43 on dist-test-slave-j4lq"
server_key: "f6a77b9b0b93af83e3ceb963b512989e"
server_key_iv: "e0d056bb34100ff9defe8c4991c45d1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:43.897522 20668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:43.907115 20668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:43.909040 20668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:43.909198 20668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:43.909425 20668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:43.909724 20668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:43.909776 20668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.909811 20668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:43.909837 20668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:43.915521 20668 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:42933
I20260926 13:07:43.915620 20796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:42933 every 8 connection(s)
I20260926 13:07:43.915895 20668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:43.917850 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20668
I20260926 13:07:43.917969 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:43.918174 2392 external_mini_cluster.cc:1428] Setting key dc8d51b121b985a9c9e493499f38b2b4
Sep 26 13:07:43 dist-test-slave-j4lq krb5kdc[20427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428063, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:43.926293 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:43.916554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:37539 (local address 127.2.86.2:58769)
0926 13:07:43.916736 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:43.916771 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:43.917317 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:43.917530 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:43.917533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:43.918217 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:43.918520 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:43.918526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.919371 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.919372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:43.920163 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:43.920165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.920578 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.920579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:43.920714 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:43.921318 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:43.921329 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:43.922976 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:43.924854 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:43.924861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:43.924862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:43.925150 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:43.925374 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:43.925375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:43.925375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:43.925472 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:43.925785 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:43.925789 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:43.925911 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 13:07:43.925984 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:07:43.926114 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260926 13:07:43.926285 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:43.916589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:58769 (local address 127.2.86.62:37539)
0926 13:07:43.916671 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:43.916673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:43.916897 (+ 224us) server_negotiation.cc:407] Connection header received
0926 13:07:43.917412 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:43.917413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:43.917424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:43.917477 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:43.918584 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.919278 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.920271 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:43.920480 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:43.920567 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:43.923080 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:43.923082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:43.923082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:43.923090 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:43.924761 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.925235 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.925236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.925237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.925304 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:43.925547 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:43.925548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:43.925549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:43.925721 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:43.925793 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:43.926057 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 13:07:43.926151 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 13:07:43.926802 20797 heartbeater.cc:347] Connected to a master server at 127.2.86.62:37539
I20260926 13:07:43.926910 20797 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:43.927100 20797 heartbeater.cc:511] Master 127.2.86.62:37539 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
I20260926 13:07:43.927510 20472 ts_manager.cc:194] Registered new tserver with Master: ef3e898e75d74134ac7dea6f330b6399 (127.2.86.2:42933)
I20260926 13:07:43.927891 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:58769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:07:43.943703 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:44.024680 20802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:44.024959 20802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:44.025002 20802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:44.026171 20802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:44.026239 20802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:44.026286 20802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:44.027791 20802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:44.028043 20802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:44.028277 20802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:07:44.033618 20802 server_base.cc:1034] running on GCE node
W20260926 13:07:44.033643 20810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.033591 20808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.033582 20807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.034070 20802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:44.034312 20802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:44.035452 20802 hybrid_clock.cc:648] HybridClock initialized: now 1790428064035440 us; error 29 us; skew 500 ppm
Sep 26 13:07:44 dist-test-slave-j4lq krb5kdc[20427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428064, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:44.038019 20802 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:07:44.038398 20802 webserver.cc:460] Webserver started at http://127.2.86.3:46271/ using document root <none> and password file <none>
I20260926 13:07:44.038650 20802 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:44.038720 20802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:44.038863 20802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:44.039885 20802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "a260115405ae40e69c7f22338168fd26"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "90b17512e12fe57f4b0459e347e3c377"
server_key_iv: "8d126b8f34a0ae014580d034f948de44"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.040243 20802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "a260115405ae40e69c7f22338168fd26"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "90b17512e12fe57f4b0459e347e3c377"
server_key_iv: "8d126b8f34a0ae014580d034f948de44"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.041615 20802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:44.042351 20817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.042528 20802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.042627 20802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "a260115405ae40e69c7f22338168fd26"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "90b17512e12fe57f4b0459e347e3c377"
server_key_iv: "8d126b8f34a0ae014580d034f948de44"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.042711 20802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:44.057166 20802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:44.059480 20802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:44.059648 20802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:44.059885 20802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:44.060204 20802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:44.060259 20802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.060300 20802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:44.060328 20802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.066212 20802 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:34771
I20260926 13:07:44.066253 20930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:34771 every 8 connection(s)
I20260926 13:07:44.066570 20802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:44.068356 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20802
I20260926 13:07:44.068470 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:44.068655 2392 external_mini_cluster.cc:1428] Setting key ba9b5f38cb05cf55612e73c96dc9e95d
Sep 26 13:07:44 dist-test-slave-j4lq krb5kdc[20427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428064, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:44.075881 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.067266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:58695 (local address 127.2.86.62:37539)
0926 13:07:44.067342 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.067344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.067498 (+ 154us) server_negotiation.cc:407] Connection header received
0926 13:07:44.068022 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.068023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.068034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.068089 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.069478 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.070032 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.070996 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.071179 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.071229 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.073263 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.073264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.073265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:44.073271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.074734 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:44.075101 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:44.075102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:44.075102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:44.075163 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:44.075370 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:44.075370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:44.075371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:44.075468 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.075516 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.075740 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.075796 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 13:07:44.076068 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.067227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:37539 (local address 127.2.86.3:58695)
0926 13:07:44.067391 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.067428 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.067896 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.068226 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.068231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.069095 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:44.069395 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.069403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.070116 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.070116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.070922 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.070924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.071247 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.071248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.071351 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.071814 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:44.071822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:44.073180 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:44.074807 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:44.074813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:44.074815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:44.075037 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:44.075220 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:44.075220 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:44.075221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:44.075316 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:44.075519 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:44.075523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:44.075636 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.075718 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.075833 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260926 13:07:44.076555 20931 heartbeater.cc:347] Connected to a master server at 127.2.86.62:37539
I20260926 13:07:44.076665 20931 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:44.076843 20931 heartbeater.cc:511] Master 127.2.86.62:37539 requested a full tablet report, sending...
I20260926 13:07:44.077271 20472 ts_manager.cc:194] Registered new tserver with Master: a260115405ae40e69c7f22338168fd26 (127.2.86.3:34771)
I20260926 13:07:44.077636 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:58695
I20260926 13:07:44.080181 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:44.086918 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.081512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.2.86.62:37539)
0926 13:07:44.081574 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.081576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.081648 (+ 72us) server_negotiation.cc:407] Connection header received
0926 13:07:44.081702 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.081703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.081711 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.081749 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.082223 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.082679 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.083339 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.083505 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.083552 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.084776 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.084777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.084777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:44.084782 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.085987 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:44.086334 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:44.086334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:44.086335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:44.086383 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:44.086528 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:44.086529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:44.086529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:44.086655 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.086735 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.086816 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.086860 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:07:44.088578 20472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:44.090355 20472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:44.096803 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.092821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.2.86.1:37295)
0926 13:07:44.093061 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.093065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.093075 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:07:44.093350 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.093354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.093373 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.093448 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.093650 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094407 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.095862 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096360 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096448 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096510 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.096664 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.096708 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":115,"threads_started":1}
I20260926 13:07:44.096887 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.092635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:42933 (local address 127.0.0.1:40846)
0926 13:07:44.092803 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.092846 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.092935 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.093392 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.093396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.093402 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.093486 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.093488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094826 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.094827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.095640 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.095643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096577 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.096654 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096658 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.096688 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.096734 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260926 13:07:44.096829 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.092658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37295 (local address 127.0.0.1:43366)
0926 13:07:44.093024 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.093034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.093087 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.093530 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.093531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.093533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.093567 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.093568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094507 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.094508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.095761 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.095763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096448 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.096523 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096543 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.096651 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.096710 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260926 13:07:44.097067 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.093008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.2.86.3:34771)
0926 13:07:44.093186 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.093190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.093231 (+ 41us) server_negotiation.cc:407] Connection header received
0926 13:07:44.093285 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.093287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.093301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.093384 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.093546 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094665 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.095891 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096583 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096641 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096696 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.096947 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.096991 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260926 13:07:44.097121 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.092739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.2.86.2:42933)
0926 13:07:44.093186 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.093190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.093199 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:44.093240 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.093242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.093259 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.093635 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.093708 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094746 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.095730 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096473 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096891 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096971 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.096994 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.097033 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":71,"threads_started":1}
I20260926 13:07:44.097249 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.092833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34771 (local address 127.0.0.1:38108)
0926 13:07:44.093176 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.093185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.093232 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.093392 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.093396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.093398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.093485 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.093487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.094766 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.094766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.095816 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.095818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.096727 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.096728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.096780 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.096783 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.096830 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.096891 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":6,"thread_start_us":98,"threads_started":1}
I20260926 13:07:44.097862 20731 tablet_service.cc:1467] Processing CreateTablet for tablet f279247672f9409698629ef1c7580fbc (DEFAULT_TABLE table=test-table [id=75ed36c3fa7c403ca00f5af9f096366e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.097920 20865 tablet_service.cc:1467] Processing CreateTablet for tablet f279247672f9409698629ef1c7580fbc (DEFAULT_TABLE table=test-table [id=75ed36c3fa7c403ca00f5af9f096366e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.098222 20731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f279247672f9409698629ef1c7580fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.098222 20865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f279247672f9409698629ef1c7580fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.098341 20597 tablet_service.cc:1467] Processing CreateTablet for tablet f279247672f9409698629ef1c7580fbc (DEFAULT_TABLE table=test-table [id=75ed36c3fa7c403ca00f5af9f096366e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.098661 20597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f279247672f9409698629ef1c7580fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.101554 20950 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Bootstrap starting.
I20260926 13:07:44.101583 20949 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Bootstrap starting.
I20260926 13:07:44.101890 20948 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Bootstrap starting.
I20260926 13:07:44.102295 20949 tablet_bootstrap.cc:654] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.102315 20950 tablet_bootstrap.cc:654] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.102561 20948 tablet_bootstrap.cc:654] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.102859 20949 log.cc:824] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.102880 20950 log.cc:824] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.103104 20948 log.cc:824] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.103694 20949 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: No bootstrap required, opened a new log
I20260926 13:07:44.103780 20949 ts_tablet_manager.cc:1397] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:44.103828 20948 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: No bootstrap required, opened a new log
I20260926 13:07:44.103847 20950 tablet_bootstrap.cc:492] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: No bootstrap required, opened a new log
I20260926 13:07:44.103901 20948 ts_tablet_manager.cc:1397] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:44.103924 20950 ts_tablet_manager.cc:1397] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:44.105418 20949 raft_consensus.cc:348] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.105599 20949 raft_consensus.cc:374] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.105669 20949 raft_consensus.cc:729] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef3e898e75d74134ac7dea6f330b6399, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.105796 20949 consensus_queue.cc:260] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.105860 20950 raft_consensus.cc:348] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.105860 20948 raft_consensus.cc:348] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.106031 20950 raft_consensus.cc:374] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.106031 20948 raft_consensus.cc:374] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.106067 20950 raft_consensus.cc:729] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a260115405ae40e69c7f22338168fd26, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.106067 20948 raft_consensus.cc:729] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bcd245cf66546118c3afda0c491fe7c, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.106098 20797 heartbeater.cc:503] Master 127.2.86.62:37539 was elected leader, sending a full tablet report...
I20260926 13:07:44.106176 20950 consensus_queue.cc:260] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.106178 20948 consensus_queue.cc:260] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.106101 20949 ts_tablet_manager.cc:1428] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:07:44.106444 20948 ts_tablet_manager.cc:1428] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:44.106441 20950 ts_tablet_manager.cc:1428] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:07:44.106549 20663 heartbeater.cc:503] Master 127.2.86.62:37539 was elected leader, sending a full tablet report...
I20260926 13:07:44.106559 20931 heartbeater.cc:503] Master 127.2.86.62:37539 was elected leader, sending a full tablet report...
W20260926 13:07:44.166486 20798 tablet.cc:2369] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:07:44.262042 20664 tablet.cc:2369] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:07:44.303586 20954 raft_consensus.cc:482] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:44.303718 20954 raft_consensus.cc:504] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.304100 20954 leader_election.cc:288] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bcd245cf66546118c3afda0c491fe7c (127.2.86.1:37295), a260115405ae40e69c7f22338168fd26 (127.2.86.3:34771)
I20260926 13:07:44.307016 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.304347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:36825 (local address 127.2.86.1:37295)
0926 13:07:44.304399 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.304401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.304413 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:07:44.304519 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.304520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.304533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.304593 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.304793 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.305316 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306277 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.306640 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306696 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.306705 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.306900 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.306942 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:07:44.307019 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.304439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:33661 (local address 127.2.86.3:34771)
0926 13:07:44.304482 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.304484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.304557 (+ 73us) server_negotiation.cc:407] Connection header received
0926 13:07:44.304622 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.304623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.304634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.304696 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.304909 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.305398 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306306 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.306663 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306755 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.306764 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.306923 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.306961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:07:44.307016 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.304304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37295 (local address 127.2.86.2:36825)
0926 13:07:44.304364 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.304376 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.304424 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.304611 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.304612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.304613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.304703 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.304705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.305399 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.305400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.306173 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.306176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.306726 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.306821 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.306824 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.306866 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.306931 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 13:07:44.307035 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.304305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:34771 (local address 127.2.86.2:33661)
0926 13:07:44.304473 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.304483 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.304552 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.304705 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.304706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.304708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.304792 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.304793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.305475 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.305476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.306194 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.306196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.306778 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.306779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.306830 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.306834 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.306907 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.306944 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":15,"thread_start_us":104,"threads_started":1}
I20260926 13:07:44.307420 20885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f279247672f9409698629ef1c7580fbc" candidate_uuid: "ef3e898e75d74134ac7dea6f330b6399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a260115405ae40e69c7f22338168fd26" is_pre_election: true
I20260926 13:07:44.307418 20617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f279247672f9409698629ef1c7580fbc" candidate_uuid: "ef3e898e75d74134ac7dea6f330b6399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcd245cf66546118c3afda0c491fe7c" is_pre_election: true
I20260926 13:07:44.307580 20617 raft_consensus.cc:2454] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef3e898e75d74134ac7dea6f330b6399 in term 0.
I20260926 13:07:44.307580 20885 raft_consensus.cc:2454] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef3e898e75d74134ac7dea6f330b6399 in term 0.
I20260926 13:07:44.307812 20685 leader_election.cc:302] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bcd245cf66546118c3afda0c491fe7c, ef3e898e75d74134ac7dea6f330b6399; no voters:
I20260926 13:07:44.307968 20954 raft_consensus.cc:2781] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:44.308053 20954 raft_consensus.cc:482] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:44.308086 20954 raft_consensus.cc:3037] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.308809 20954 raft_consensus.cc:504] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.309000 20954 leader_election.cc:288] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [CANDIDATE]: Term 1 election: Requested vote from peers 2bcd245cf66546118c3afda0c491fe7c (127.2.86.1:37295), a260115405ae40e69c7f22338168fd26 (127.2.86.3:34771)
I20260926 13:07:44.309178 20885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f279247672f9409698629ef1c7580fbc" candidate_uuid: "ef3e898e75d74134ac7dea6f330b6399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a260115405ae40e69c7f22338168fd26"
I20260926 13:07:44.309221 20617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f279247672f9409698629ef1c7580fbc" candidate_uuid: "ef3e898e75d74134ac7dea6f330b6399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcd245cf66546118c3afda0c491fe7c"
I20260926 13:07:44.309283 20885 raft_consensus.cc:3037] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.309290 20617 raft_consensus.cc:3037] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.310020 20617 raft_consensus.cc:2454] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef3e898e75d74134ac7dea6f330b6399 in term 1.
I20260926 13:07:44.310035 20885 raft_consensus.cc:2454] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef3e898e75d74134ac7dea6f330b6399 in term 1.
I20260926 13:07:44.310209 20684 leader_election.cc:302] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a260115405ae40e69c7f22338168fd26, ef3e898e75d74134ac7dea6f330b6399; no voters:
I20260926 13:07:44.310322 20954 raft_consensus.cc:2781] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:44.310482 20954 raft_consensus.cc:686] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 1 LEADER]: Becoming Leader. State: Replica: ef3e898e75d74134ac7dea6f330b6399, State: Running, Role: LEADER
I20260926 13:07:44.310583 20954 consensus_queue.cc:237] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } }
I20260926 13:07:44.311470 20472 catalog_manager.cc:5495] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef3e898e75d74134ac7dea6f330b6399 (127.2.86.2). New cstate: current_term: 1 leader_uuid: "ef3e898e75d74134ac7dea6f330b6399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef3e898e75d74134ac7dea6f330b6399" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 } health_report { overall_health: UNKNOWN } } }
W20260926 13:07:44.317061 20932 tablet.cc:2369] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:07:44.327693 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.325461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.2.86.2:42933)
0926 13:07:44.325514 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.325516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.325527 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:44.325604 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.325604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.325622 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.325684 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:44.325787 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.326247 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.326994 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.327182 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.327230 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.327365 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:44.327450 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:44.327520 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.327572 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.327579 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:07:44.329506 20617 raft_consensus.cc:1261] T f279247672f9409698629ef1c7580fbc P 2bcd245cf66546118c3afda0c491fe7c [term 1 FOLLOWER]: Refusing update from remote peer ef3e898e75d74134ac7dea6f330b6399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:44.329532 20885 raft_consensus.cc:1261] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 1 FOLLOWER]: Refusing update from remote peer ef3e898e75d74134ac7dea6f330b6399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:44.329846 20958 consensus_queue.cc:1035] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a260115405ae40e69c7f22338168fd26" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:44.329993 20954 consensus_queue.cc:1035] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bcd245cf66546118c3afda0c491fe7c" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:44.332278 20961 mvcc.cc:205] Tried to move back new op lower bound from 7333593351491100672 to 7333593351416766464. Current Snapshot: MvccSnapshot[applied={T|T < 7333593351491100672}]
I20260926 13:07:44.332970 20963 mvcc.cc:205] Tried to move back new op lower bound from 7333593351491100672 to 7333593351416766464. Current Snapshot: MvccSnapshot[applied={T|T < 7333593351491100672}]
I20260926 13:07:44.332971 20962 mvcc.cc:205] Tried to move back new op lower bound from 7333593351491100672 to 7333593351416766464. Current Snapshot: MvccSnapshot[applied={T|T < 7333593351491100672}]
I20260926 13:07:44.335814 20472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:44.335924 20472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:44.336735 20472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5689d69157e84b8c870a48158ae29a93: Sending DeleteTablet for 3 replicas of tablet f279247672f9409698629ef1c7580fbc
I20260926 13:07:44.337141 20597 tablet_service.cc:1514] Processing DeleteTablet for tablet f279247672f9409698629ef1c7580fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:43366
I20260926 13:07:44.337164 20731 tablet_service.cc:1514] Processing DeleteTablet for tablet f279247672f9409698629ef1c7580fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:40846
I20260926 13:07:44.337221 20865 tablet_service.cc:1514] Processing DeleteTablet for tablet f279247672f9409698629ef1c7580fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:38108
I20260926 13:07:44.337473 20972 tablet_replica.cc:330] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: stopping tablet replica
I20260926 13:07:44.337590 20972 raft_consensus.cc:2229] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:44.337608 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20534
I20260926 13:07:44.338847 20972 raft_consensus.cc:2258] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:44.339401 20972 ts_tablet_manager.cc:1905] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:44.341149 20972 ts_tablet_manager.cc:1918] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:44.341233 20972 log.cc:1196] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/f279247672f9409698629ef1c7580fbc
I20260926 13:07:44.341580 20972 ts_tablet_manager.cc:1939] T f279247672f9409698629ef1c7580fbc P a260115405ae40e69c7f22338168fd26: Deleting consensus metadata
I20260926 13:07:44.342336 20456 catalog_manager.cc:4841] TS a260115405ae40e69c7f22338168fd26 (127.2.86.3:34771): tablet f279247672f9409698629ef1c7580fbc (table test-table [id=75ed36c3fa7c403ca00f5af9f096366e]) successfully deleted
W20260926 13:07:44.342960 20457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:37295 (error 108)
I20260926 13:07:44.343302 20970 tablet_replica.cc:330] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: stopping tablet replica
I20260926 13:07:44.343422 20970 raft_consensus.cc:2229] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:07:44.343575 20970 raft_consensus.cc:2258] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 13:07:44.344002 20945 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:07:44.343401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37295 (local address 127.0.0.1:43376)
0926 13:07:44.343470 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.343483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.343535 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.343954 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:37295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 13:07:44.344053 20970 ts_tablet_manager.cc:1905] T f279247672f9409698629ef1c7580fbc P ef3e898e75d74134ac7dea6f330b6399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:44.344089 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20668
W20260926 13:07:44.344261 20457 catalog_manager.cc:4584] TS 2bcd245cf66546118c3afda0c491fe7c (127.2.86.1:37295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f279247672f9409698629ef1c7580fbc: Network error: Client connection negotiation failed: client connection to 127.2.86.1:37295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:07:44.349486 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20802
I20260926 13:07:44.349597 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.349428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:42933 (local address 127.0.0.1:40860)
0926 13:07:44.349487 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.349536 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.2:42933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:07:44.355033 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20440
2026-09-26T13:07:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2058 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T13:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:44Z Disabled control of system clock
I20260926 13:07:44.382323 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38321
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:44573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:38321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:44.462159 20977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:44.462368 20977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:44.462388 20977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:44.463482 20977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:44.463518 20977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:44.463531 20977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:44.463543 20977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:44.463557 20977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:44.464821 20977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:38321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:38321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:44.465090 20977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:44.465329 20977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:44.469902 20983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.469902 20985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.469882 20982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.470208 20977 server_base.cc:1034] running on GCE node
I20260926 13:07:44.470429 20977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:44.470734 20977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:44.471871 20977 hybrid_clock.cc:648] HybridClock initialized: now 1790428064471854 us; error 33 us; skew 500 ppm
I20260926 13:07:44.473155 20977 webserver.cc:460] Webserver started at http://127.2.86.62:41673/ using document root <none> and password file <none>
I20260926 13:07:44.473409 20977 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:44.473455 20977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:44.473598 20977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:44.474706 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "b009c5af6cb949a8bcf46b95621df42c"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "ba9a8e992a5d9962e8e80a08b8e1ee30"
server_key_iv: "dd630aa79a0e220520cb4088f73e5e1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.475054 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "b009c5af6cb949a8bcf46b95621df42c"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "ba9a8e992a5d9962e8e80a08b8e1ee30"
server_key_iv: "dd630aa79a0e220520cb4088f73e5e1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.476320 20977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:44.477043 20991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.477217 20977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:44.477309 20977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "b009c5af6cb949a8bcf46b95621df42c"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "ba9a8e992a5d9962e8e80a08b8e1ee30"
server_key_iv: "dd630aa79a0e220520cb4088f73e5e1e"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.477387 20977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:44.503942 20977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:44.504279 20977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:44.504437 20977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:44.508213 20977 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:38321
I20260926 13:07:44.508268 21043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:38321 every 8 connection(s)
I20260926 13:07:44.508613 20977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:44.509617 21044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.511785 21044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: Bootstrap starting.
I20260926 13:07:44.512313 21044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.512547 21044 log.cc:824] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.513217 21044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: No bootstrap required, opened a new log
I20260926 13:07:44.514703 21044 raft_consensus.cc:348] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } }
I20260926 13:07:44.514835 21044 raft_consensus.cc:374] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.514858 21044 raft_consensus.cc:729] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b009c5af6cb949a8bcf46b95621df42c, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.514999 21044 consensus_queue.cc:260] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } }
I20260926 13:07:44.515089 21044 raft_consensus.cc:388] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:44.515111 21044 raft_consensus.cc:482] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:44.515139 21044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.515596 21044 raft_consensus.cc:504] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } }
I20260926 13:07:44.515677 21044 leader_election.cc:302] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b009c5af6cb949a8bcf46b95621df42c; no voters:
I20260926 13:07:44.515805 21044 leader_election.cc:288] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:44.515877 21047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:44.516047 21047 raft_consensus.cc:686] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [term 1 LEADER]: Becoming Leader. State: Replica: b009c5af6cb949a8bcf46b95621df42c, State: Running, Role: LEADER
I20260926 13:07:44.516054 21044 sys_catalog.cc:564] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:44.516170 21047 consensus_queue.cc:237] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } }
I20260926 13:07:44.516503 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 20977
I20260926 13:07:44.516517 21049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b009c5af6cb949a8bcf46b95621df42c. Latest consensus state: current_term: 1 leader_uuid: "b009c5af6cb949a8bcf46b95621df42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } } }
I20260926 13:07:44.516609 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:44.516601 21049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:44.516602 21048 sys_catalog.cc:455] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b009c5af6cb949a8bcf46b95621df42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b009c5af6cb949a8bcf46b95621df42c" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 38321 } } }
I20260926 13:07:44.516680 21048 sys_catalog.cc:458] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:44.516811 21051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:44.516829 2392 external_mini_cluster.cc:1428] Setting key 90b0a4b30077b348c2c2202292cbc41a
I20260926 13:07:44.517335 21051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:44.518810 21051 catalog_manager.cc:1347] Generated new cluster ID: cde19d3d2a914477a58d8b2efb4146be
I20260926 13:07:44.518855 21051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:44.522565 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.517505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.2.86.62:38321)
0926 13:07:44.518260 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.518268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.518303 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:07:44.518423 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.518429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.518498 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.518647 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.518865 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.519730 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.520583 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.521076 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.521208 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.521912 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.521916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.521916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:44.522105 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.522138 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:44.522149 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.522222 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.522293 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.522369 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":61,"threads_started":1}
I20260926 13:07:44.531392 21051 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:44.532056 21051 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:44.539255 21051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: Generated new TSK 0
I20260926 13:07:44.539433 21051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:44.543852 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--builtin_ntp_servers=127.2.86.20:44573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:44.633631 21068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:44.633867 21068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:44.633901 21068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:44.635058 21068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:44.635121 21068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:44.635165 21068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:44.636514 21068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:44.636771 21068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:44.637100 21068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:44.642765 21073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.642784 21074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.642825 21076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.643061 21068 server_base.cc:1034] running on GCE node
I20260926 13:07:44.643308 21068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:44.643590 21068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:44.644764 21068 hybrid_clock.cc:648] HybridClock initialized: now 1790428064644736 us; error 45 us; skew 500 ppm
I20260926 13:07:44.646121 21068 webserver.cc:460] Webserver started at http://127.2.86.1:33629/ using document root <none> and password file <none>
I20260926 13:07:44.646358 21068 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:44.646423 21068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:44.646566 21068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:44.647603 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "07492e8edd754cc9958a40b19b8d93d3"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "f3c7e75eb968531f1da21664e9c5f20c"
server_key_iv: "820f766db88d45d768f8e60ba8397e6b"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.647959 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "07492e8edd754cc9958a40b19b8d93d3"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "f3c7e75eb968531f1da21664e9c5f20c"
server_key_iv: "820f766db88d45d768f8e60ba8397e6b"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.649410 21068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:44.650183 21082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.650332 21068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:44.650413 21068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "07492e8edd754cc9958a40b19b8d93d3"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "f3c7e75eb968531f1da21664e9c5f20c"
server_key_iv: "820f766db88d45d768f8e60ba8397e6b"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.650477 21068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:44.664978 21068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:44.665226 21068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:44.665324 21068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:44.665491 21068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:44.665752 21068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:44.665781 21068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.665802 21068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:44.665814 21068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.671475 21068 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:38279
I20260926 13:07:44.671519 21195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:38279 every 8 connection(s)
I20260926 13:07:44.671823 21068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:07:44.676589 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.672460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:34177 (local address 127.2.86.62:38321)
0926 13:07:44.672544 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.672546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.672809 (+ 263us) server_negotiation.cc:407] Connection header received
0926 13:07:44.673275 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.673276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.673287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.673337 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.673952 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.674425 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.675325 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.675503 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.675553 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.676382 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.676383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.676383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:44.676386 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.676399 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:44.676402 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.676442 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.676505 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.676525 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 13:07:44.676838 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.672447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38321 (local address 127.2.86.1:34177)
0926 13:07:44.672682 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.672715 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.673189 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.673395 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.673398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.673405 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:44.673879 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.673883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.674501 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.674501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.675250 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.675252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.675573 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.675574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.675682 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.676261 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:44.676262 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:44.676289 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:44.676310 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:44.676441 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:44.676443 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:44.676445 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.676491 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.676560 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260926 13:07:44.677345 21196 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38321
I20260926 13:07:44.677446 21196 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:44.677626 21196 heartbeater.cc:511] Master 127.2.86.62:38321 requested a full tablet report, sending...
I20260926 13:07:44.678085 21008 ts_manager.cc:194] Registered new tserver with Master: 07492e8edd754cc9958a40b19b8d93d3 (127.2.86.1:38279)
I20260926 13:07:44.678670 21008 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.1:34177
I20260926 13:07:44.679171 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21068
I20260926 13:07:44.679245 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:44.679463 2392 external_mini_cluster.cc:1428] Setting key d9edcd749342793537883c4ec3efd826
I20260926 13:07:44.680281 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--builtin_ntp_servers=127.2.86.20:44573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:44.760298 21199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:44.760499 21199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:44.760520 21199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:44.761899 21199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:44.761952 21199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:44.761984 21199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:44.763300 21199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:44.763490 21199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:44.763739 21199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:44.768666 21205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.768656 21204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.768932 21199 server_base.cc:1034] running on GCE node
W20260926 13:07:44.768651 21207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.769183 21199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:44.769423 21199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:44.770582 21199 hybrid_clock.cc:648] HybridClock initialized: now 1790428064770571 us; error 27 us; skew 500 ppm
I20260926 13:07:44.771864 21199 webserver.cc:460] Webserver started at http://127.2.86.2:39049/ using document root <none> and password file <none>
I20260926 13:07:44.772115 21199 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:44.772188 21199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:44.772332 21199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:44.773453 21199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "dce4a37d63ab439cb248a7a52fdc1172"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "bfc1e2b8217d8b45d87e31a58b8ef105"
server_key_iv: "3c7decd376d42eac1572f954b94936d8"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.773813 21199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "dce4a37d63ab439cb248a7a52fdc1172"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "bfc1e2b8217d8b45d87e31a58b8ef105"
server_key_iv: "3c7decd376d42eac1572f954b94936d8"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.775074 21199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:44.775765 21213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.775928 21199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:44.776028 21199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "dce4a37d63ab439cb248a7a52fdc1172"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "bfc1e2b8217d8b45d87e31a58b8ef105"
server_key_iv: "3c7decd376d42eac1572f954b94936d8"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.776099 21199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:44.788898 21199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:44.789204 21199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:44.789356 21199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:44.789570 21199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:44.789898 21199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:44.789954 21199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.789994 21199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:44.790023 21199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.795492 21199 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:32869
I20260926 13:07:44.795603 21326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:32869 every 8 connection(s)
I20260926 13:07:44.795894 21199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:44.801169 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.796616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:43261 (local address 127.2.86.62:38321)
0926 13:07:44.796694 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.796696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.796976 (+ 280us) server_negotiation.cc:407] Connection header received
0926 13:07:44.797496 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.797498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.797513 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.797576 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.798235 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.798733 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.799703 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.799910 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.799993 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.800847 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.800849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.800849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:44.800854 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.800891 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:44.800895 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.800940 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.801037 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.801065 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 13:07:44.801348 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.796636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38321 (local address 127.2.86.2:43261)
0926 13:07:44.796826 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.796858 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.797388 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.797655 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.797658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.797666 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:44.798139 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.798143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.798827 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.798828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.799590 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.799591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.800002 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.800003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.800132 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.800702 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:44.800702 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:44.800729 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:44.800750 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:44.800940 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:44.800945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:44.800948 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.801040 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.801138 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260926 13:07:44.801826 21327 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38321
I20260926 13:07:44.801939 21327 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:44.802114 21327 heartbeater.cc:511] Master 127.2.86.62:38321 requested a full tablet report, sending...
I20260926 13:07:44.802496 21008 ts_manager.cc:194] Registered new tserver with Master: dce4a37d63ab439cb248a7a52fdc1172 (127.2.86.2:32869)
I20260926 13:07:44.802850 21008 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.2:43261
I20260926 13:07:44.804860 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21199
I20260926 13:07:44.804989 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:44.805191 2392 external_mini_cluster.cc:1428] Setting key 95ebc8920b57a16ff2541b8fa1a4db2f
I20260926 13:07:44.805963 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--builtin_ntp_servers=127.2.86.20:44573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:44.885612 21330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:44.885797 21330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:44.885818 21330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:44.886943 21330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:44.886982 21330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:44.887032 21330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:44.888345 21330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:38321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:44.888543 21330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:44.888813 21330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:44.893680 21336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.893694 21338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:44.893690 21335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:44.893780 21330 server_base.cc:1034] running on GCE node
I20260926 13:07:44.894129 21330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:44.894376 21330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:44.895535 21330 hybrid_clock.cc:648] HybridClock initialized: now 1790428064895523 us; error 26 us; skew 500 ppm
I20260926 13:07:44.896734 21330 webserver.cc:460] Webserver started at http://127.2.86.3:33083/ using document root <none> and password file <none>
I20260926 13:07:44.897022 21330 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:44.897104 21330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:44.897264 21330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:44.898303 21330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "410f08fd3b7d41eeb3138f6b64df72b0"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "21d497523f804e6c41488339eddfea7c"
server_key_iv: "372ab7b641e5a366393823c7abc00146"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.898649 21330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "410f08fd3b7d41eeb3138f6b64df72b0"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "21d497523f804e6c41488339eddfea7c"
server_key_iv: "372ab7b641e5a366393823c7abc00146"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.899900 21330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:44.900606 21344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.900792 21330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:44.900904 21330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "410f08fd3b7d41eeb3138f6b64df72b0"
format_stamp: "Formatted at 2026-09-26 13:07:44 on dist-test-slave-j4lq"
server_key: "21d497523f804e6c41488339eddfea7c"
server_key_iv: "372ab7b641e5a366393823c7abc00146"
server_key_version: "encryptionkey@0"
I20260926 13:07:44.900980 21330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:44.921797 21330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:44.922130 21330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:44.922312 21330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:44.922549 21330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:44.922887 21330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:44.922946 21330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.922989 21330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:44.923020 21330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:44.928628 21330 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:35565
I20260926 13:07:44.928733 21457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:35565 every 8 connection(s)
I20260926 13:07:44.929090 21330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:44.930480 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21330
I20260926 13:07:44.930595 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:44.930828 2392 external_mini_cluster.cc:1428] Setting key 0bfebd7815aa64466b62a913c7f5c056
I20260926 13:07:44.934468 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.929811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:59657 (local address 127.2.86.62:38321)
0926 13:07:44.929878 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.929879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.930085 (+ 206us) server_negotiation.cc:407] Connection header received
0926 13:07:44.930617 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.930618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.930630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.930699 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.931664 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.932178 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.933200 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.933379 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.933429 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.934201 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.934202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.934202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:44.934206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.934219 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:44.934223 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.934265 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.934386 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.934410 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:07:44.934672 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.929803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:38321 (local address 127.2.86.3:59657)
0926 13:07:44.929980 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.930013 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.930503 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.930794 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.930799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.930810 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:44.931534 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.931541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.932254 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.932254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.933129 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.933131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.933450 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.933451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.933554 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.934056 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:44.934057 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:44.934088 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:44.934111 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:44.934299 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:44.934305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:44.934308 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.934374 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.934456 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260926 13:07:44.935134 21458 heartbeater.cc:347] Connected to a master server at 127.2.86.62:38321
I20260926 13:07:44.935232 21458 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:44.935370 21458 heartbeater.cc:511] Master 127.2.86.62:38321 requested a full tablet report, sending...
I20260926 13:07:44.935693 21008 ts_manager.cc:194] Registered new tserver with Master: 410f08fd3b7d41eeb3138f6b64df72b0 (127.2.86.3:35565)
I20260926 13:07:44.936008 21008 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.3:59657
I20260926 13:07:44.942682 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:44.947096 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.944015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.2.86.62:38321)
0926 13:07:44.944099 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.944100 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.944183 (+ 83us) server_negotiation.cc:407] Connection header received
0926 13:07:44.944260 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.944260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.944270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.944342 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:44.944523 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.945063 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.945845 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.946034 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.946127 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.946796 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:44.946797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:44.946798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:44.946803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:44.946816 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:44.946820 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:44.946931 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.946994 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.947005 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:07:44.948576 21008 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:44.950153 21008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:44.956553 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.2.86.1:38279)
0926 13:07:44.952620 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.952625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.953065 (+ 440us) server_negotiation.cc:407] Connection header received
0926 13:07:44.953111 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.953113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.953129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.953214 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.953534 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954597 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.955645 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956150 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956238 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956332 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.956419 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.956457 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260926 13:07:44.956595 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:35565 (local address 127.0.0.1:36146)
0926 13:07:44.952652 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.952690 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.952785 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.953029 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.953032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.953034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.953108 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.953110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954212 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.954214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.955441 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.955444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956327 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.956389 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956394 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.956424 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.956467 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":87,"threads_started":1}
I20260926 13:07:44.956799 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38279 (local address 127.0.0.1:42592)
0926 13:07:44.952936 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.952946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.952994 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.953364 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.953364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.953367 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.953401 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.953402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954694 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.954695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.955559 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.955561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956240 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.956314 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956326 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.956402 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.956455 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260926 13:07:44.956817 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.2.86.3:35565)
0926 13:07:44.952589 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.952595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.952758 (+ 163us) server_negotiation.cc:407] Connection header received
0926 13:07:44.952902 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.952904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.952920 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.953025 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.953254 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954129 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.955533 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956228 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956586 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956667 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.956690 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.956728 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260926 13:07:44.957144 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.2.86.2:32869)
0926 13:07:44.952770 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.952774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.953067 (+ 293us) server_negotiation.cc:407] Connection header received
0926 13:07:44.953111 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.953113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.953129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.953203 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.953396 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954570 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.955771 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956470 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956625 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.956986 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.957035 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":99,"threads_started":1}
I20260926 13:07:44.956910 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.952244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32869 (local address 127.0.0.1:60102)
0926 13:07:44.952948 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.952958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.953005 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.953267 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.953268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.953270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.953322 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.953323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.954648 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.954649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.955682 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.955684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.956659 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.956660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.956718 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.956721 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.956771 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.956806 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":80,"threads_started":1}
I20260926 13:07:44.957597 21392 tablet_service.cc:1467] Processing CreateTablet for tablet 314d05b697a541dab14ec8ad237a89d4 (DEFAULT_TABLE table=test-table [id=3e41d89f74d44f15b440cca1db3a1f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.957778 21130 tablet_service.cc:1467] Processing CreateTablet for tablet 314d05b697a541dab14ec8ad237a89d4 (DEFAULT_TABLE table=test-table [id=3e41d89f74d44f15b440cca1db3a1f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.957964 21392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314d05b697a541dab14ec8ad237a89d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.958074 21130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314d05b697a541dab14ec8ad237a89d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.961314 21472 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: Bootstrap starting.
I20260926 13:07:44.961536 21473 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Bootstrap starting.
I20260926 13:07:44.961525 21261 tablet_service.cc:1467] Processing CreateTablet for tablet 314d05b697a541dab14ec8ad237a89d4 (DEFAULT_TABLE table=test-table [id=3e41d89f74d44f15b440cca1db3a1f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:44.961882 21261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314d05b697a541dab14ec8ad237a89d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:44.962206 21473 tablet_bootstrap.cc:654] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.962533 21473 log.cc:824] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.963240 21473 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: No bootstrap required, opened a new log
I20260926 13:07:44.963325 21473 ts_tablet_manager.cc:1397] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:44.963367 21472 tablet_bootstrap.cc:654] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.963785 21472 log.cc:824] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.964597 21472 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: No bootstrap required, opened a new log
I20260926 13:07:44.964680 21472 ts_tablet_manager.cc:1397] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 13:07:44.965323 21476 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Bootstrap starting.
I20260926 13:07:44.965998 21476 tablet_bootstrap.cc:654] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:44.966058 21473 raft_consensus.cc:348] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.966239 21473 raft_consensus.cc:374] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.966275 21473 raft_consensus.cc:729] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410f08fd3b7d41eeb3138f6b64df72b0, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.966369 21476 log.cc:824] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:44.966430 21473 consensus_queue.cc:260] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.966697 21473 ts_tablet_manager.cc:1428] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 13:07:44.966823 21458 heartbeater.cc:503] Master 127.2.86.62:38321 was elected leader, sending a full tablet report...
I20260926 13:07:44.966998 21472 raft_consensus.cc:348] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.967337 21476 tablet_bootstrap.cc:492] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: No bootstrap required, opened a new log
I20260926 13:07:44.967389 21476 ts_tablet_manager.cc:1397] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:44.967165 21472 raft_consensus.cc:374] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.967581 21472 raft_consensus.cc:729] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07492e8edd754cc9958a40b19b8d93d3, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.967713 21472 consensus_queue.cc:260] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.968040 21472 ts_tablet_manager.cc:1428] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:44.968181 21196 heartbeater.cc:503] Master 127.2.86.62:38321 was elected leader, sending a full tablet report...
I20260926 13:07:44.969799 21476 raft_consensus.cc:348] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.969941 21476 raft_consensus.cc:374] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:44.969985 21476 raft_consensus.cc:729] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce4a37d63ab439cb248a7a52fdc1172, State: Initialized, Role: FOLLOWER
I20260926 13:07:44.970094 21476 consensus_queue.cc:260] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.970328 21476 ts_tablet_manager.cc:1428] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 13:07:44.970407 21327 heartbeater.cc:503] Master 127.2.86.62:38321 was elected leader, sending a full tablet report...
I20260926 13:07:44.977272 21478 raft_consensus.cc:482] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:44.977388 21478 raft_consensus.cc:504] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.977723 21478 leader_election.cc:288] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07492e8edd754cc9958a40b19b8d93d3 (127.2.86.1:38279), dce4a37d63ab439cb248a7a52fdc1172 (127.2.86.2:32869)
I20260926 13:07:44.981005 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.977974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:42699 (local address 127.2.86.2:32869)
0926 13:07:44.978017 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.978019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.978030 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:44.978087 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.978088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.978097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.978153 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.978309 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.978951 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980200 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.980657 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980737 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.980747 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.980872 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.980937 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:07:44.981029 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.977891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:32869 (local address 127.2.86.3:42699)
0926 13:07:44.977953 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.977965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.978017 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.978166 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.978167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.978168 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.978242 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.978244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.979034 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.979035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.980126 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.980129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.980735 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.980804 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.980807 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.980859 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.980936 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":35}
I20260926 13:07:44.981104 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.977959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:36485 (local address 127.2.86.1:38279)
0926 13:07:44.977998 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.978000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.978130 (+ 130us) server_negotiation.cc:407] Connection header received
0926 13:07:44.978204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.978204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.978218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.978268 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:44.978441 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.978872 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980237 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.980657 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980778 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.980788 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.980956 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.980993 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:07:44.981349 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314d05b697a541dab14ec8ad237a89d4" candidate_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce4a37d63ab439cb248a7a52fdc1172" is_pre_election: true
I20260926 13:07:44.981485 21281 raft_consensus.cc:2454] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410f08fd3b7d41eeb3138f6b64df72b0 in term 0.
I20260926 13:07:44.981500 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.977903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38279 (local address 127.2.86.3:36485)
0926 13:07:44.978056 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:44.978066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:44.978125 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:44.978301 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:44.978302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:44.978303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:44.978366 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.978368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.978939 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.978940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.980133 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:44.980136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.980763 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.980764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:44.980859 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.980864 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:44.980949 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 13:07:44.980985 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260926 13:07:44.981735 21345 leader_election.cc:302] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410f08fd3b7d41eeb3138f6b64df72b0, dce4a37d63ab439cb248a7a52fdc1172; no voters:
I20260926 13:07:44.981832 21150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314d05b697a541dab14ec8ad237a89d4" candidate_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07492e8edd754cc9958a40b19b8d93d3" is_pre_election: true
I20260926 13:07:44.981959 21150 raft_consensus.cc:2454] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410f08fd3b7d41eeb3138f6b64df72b0 in term 0.
I20260926 13:07:44.982041 21478 raft_consensus.cc:2781] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:44.982086 21478 raft_consensus.cc:482] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:44.982125 21478 raft_consensus.cc:3037] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.982800 21478 raft_consensus.cc:504] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.982941 21478 leader_election.cc:288] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [CANDIDATE]: Term 1 election: Requested vote from peers 07492e8edd754cc9958a40b19b8d93d3 (127.2.86.1:38279), dce4a37d63ab439cb248a7a52fdc1172 (127.2.86.2:32869)
I20260926 13:07:44.983095 21281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314d05b697a541dab14ec8ad237a89d4" candidate_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce4a37d63ab439cb248a7a52fdc1172"
I20260926 13:07:44.983091 21150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314d05b697a541dab14ec8ad237a89d4" candidate_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07492e8edd754cc9958a40b19b8d93d3"
I20260926 13:07:44.983165 21281 raft_consensus.cc:3037] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.983165 21150 raft_consensus.cc:3037] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:44.984017 21281 raft_consensus.cc:2454] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410f08fd3b7d41eeb3138f6b64df72b0 in term 1.
I20260926 13:07:44.984017 21150 raft_consensus.cc:2454] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410f08fd3b7d41eeb3138f6b64df72b0 in term 1.
I20260926 13:07:44.984170 21345 leader_election.cc:302] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410f08fd3b7d41eeb3138f6b64df72b0, dce4a37d63ab439cb248a7a52fdc1172; no voters:
I20260926 13:07:44.984259 21478 raft_consensus.cc:2781] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:44.984393 21478 raft_consensus.cc:686] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 1 LEADER]: Becoming Leader. State: Replica: 410f08fd3b7d41eeb3138f6b64df72b0, State: Running, Role: LEADER
I20260926 13:07:44.984485 21478 consensus_queue.cc:237] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } }
I20260926 13:07:44.985311 21007 catalog_manager.cc:5495] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 410f08fd3b7d41eeb3138f6b64df72b0 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "410f08fd3b7d41eeb3138f6b64df72b0" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 } health_report { overall_health: UNKNOWN } } }
I20260926 13:07:44.991415 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:44.988976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.2.86.3:35565)
0926 13:07:44.989020 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:44.989022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:44.989033 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:44.989110 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:44.989111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:44.989129 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:44.989246 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:44.989396 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.989854 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.990618 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:44.990830 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:44.990907 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:44.991065 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:44.991161 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:44.991269 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:44.991303 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 13:07:44.991312 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:07:44.993114 21150 raft_consensus.cc:1261] T 314d05b697a541dab14ec8ad237a89d4 P 07492e8edd754cc9958a40b19b8d93d3 [term 1 FOLLOWER]: Refusing update from remote peer 410f08fd3b7d41eeb3138f6b64df72b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:44.993121 21281 raft_consensus.cc:1261] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 1 FOLLOWER]: Refusing update from remote peer 410f08fd3b7d41eeb3138f6b64df72b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:44.993472 21482 consensus_queue.cc:1035] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07492e8edd754cc9958a40b19b8d93d3" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:44.993600 21478 consensus_queue.cc:1035] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce4a37d63ab439cb248a7a52fdc1172" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 32869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:44.996239 21485 mvcc.cc:205] Tried to move back new op lower bound from 7333593354209247232 to 7333593354177241088. Current Snapshot: MvccSnapshot[applied={T|T < 7333593354209247232}]
I20260926 13:07:44.996882 21486 mvcc.cc:205] Tried to move back new op lower bound from 7333593354209247232 to 7333593354177241088. Current Snapshot: MvccSnapshot[applied={T|T < 7333593354209247232}]
I20260926 13:07:44.998683 21008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:44.998781 21008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:44.999521 21008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b009c5af6cb949a8bcf46b95621df42c: Sending DeleteTablet for 3 replicas of tablet 314d05b697a541dab14ec8ad237a89d4
I20260926 13:07:44.999836 21130 tablet_service.cc:1514] Processing DeleteTablet for tablet 314d05b697a541dab14ec8ad237a89d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='slave'} at 127.0.0.1:42592
I20260926 13:07:44.999874 21261 tablet_service.cc:1514] Processing DeleteTablet for tablet 314d05b697a541dab14ec8ad237a89d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='slave'} at 127.0.0.1:60102
I20260926 13:07:45.000118 21496 tablet_replica.cc:330] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: stopping tablet replica
I20260926 13:07:45.000221 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21068
I20260926 13:07:45.000213 21392 tablet_service.cc:1514] Processing DeleteTablet for tablet 314d05b697a541dab14ec8ad237a89d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:44 UTC) from {username='slave'} at 127.0.0.1:36146
I20260926 13:07:45.000278 21496 raft_consensus.cc:2229] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:45.002243 21496 raft_consensus.cc:2258] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:45.002460 21497 tablet_replica.cc:330] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: stopping tablet replica
I20260926 13:07:45.002741 21497 raft_consensus.cc:2229] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:07:45.002831 21496 ts_tablet_manager.cc:1905] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:45.002980 21497 raft_consensus.cc:2258] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:45.003429 21497 ts_tablet_manager.cc:1905] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:45.004695 21496 ts_tablet_manager.cc:1918] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:45.004778 21496 log.cc:1196] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/314d05b697a541dab14ec8ad237a89d4
I20260926 13:07:45.005065 21496 ts_tablet_manager.cc:1939] T 314d05b697a541dab14ec8ad237a89d4 P dce4a37d63ab439cb248a7a52fdc1172: Deleting consensus metadata
I20260926 13:07:45.005162 21497 ts_tablet_manager.cc:1918] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:45.005240 21497 log.cc:1196] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/314d05b697a541dab14ec8ad237a89d4
I20260926 13:07:45.005545 21497 ts_tablet_manager.cc:1939] T 314d05b697a541dab14ec8ad237a89d4 P 410f08fd3b7d41eeb3138f6b64df72b0: Deleting consensus metadata
I20260926 13:07:45.005528 20992 catalog_manager.cc:4841] TS dce4a37d63ab439cb248a7a52fdc1172 (127.2.86.2:32869): tablet 314d05b697a541dab14ec8ad237a89d4 (table test-table [id=3e41d89f74d44f15b440cca1db3a1f1b]) successfully deleted
I20260926 13:07:45.005963 20995 catalog_manager.cc:4841] TS 410f08fd3b7d41eeb3138f6b64df72b0 (127.2.86.3:35565): tablet 314d05b697a541dab14ec8ad237a89d4 (table test-table [id=3e41d89f74d44f15b440cca1db3a1f1b]) successfully deleted
W20260926 13:07:45.015892 20994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:38279 (error 108)
I20260926 13:07:45.016486 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:45.016338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:38279 (local address 127.0.0.1:42600)
0926 13:07:45.016402 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:45.016450 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:38279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 13:07:45.016563 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21199
W20260926 13:07:45.017920 20994 catalog_manager.cc:4584] TS 07492e8edd754cc9958a40b19b8d93d3 (127.2.86.1:38279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 314d05b697a541dab14ec8ad237a89d4: Network error: Client connection negotiation failed: client connection to 127.2.86.1:38279: connect: Connection refused (error 111)
I20260926 13:07:45.021158 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21330
I20260926 13:07:45.025892 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 20977
2026-09-26T13:07:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (668 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:45 dist-test-slave-j4lq krb5kdc[21501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:45 dist-test-slave-j4lq krb5kdc[21501](info): set up 2 sockets
Sep 26 13:07:45 dist-test-slave-j4lq krb5kdc[21501](info): commencing operation
krb5kdc: starting...
W20260926 13:07:46.052938 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:07:46.163559 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40607
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:40607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:46.243403 21514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:46.243639 21514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:46.243676 21514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:46.244818 21514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:46.244910 21514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:46.244943 21514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:46.244972 21514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:46.244997 21514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:46.246462 21514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:40607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:40607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:46.246735 21514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:46.246959 21514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:46.251585 21522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.251609 21519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.251585 21520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:46.251858 21514 server_base.cc:1034] running on GCE node
I20260926 13:07:46.252082 21514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:46.252399 21514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:46.253556 21514 hybrid_clock.cc:648] HybridClock initialized: now 1790428066253545 us; error 38 us; skew 500 ppm
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:46.256057 21514 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:07:46.256515 21514 webserver.cc:460] Webserver started at http://127.2.86.62:35111/ using document root <none> and password file <none>
I20260926 13:07:46.256738 21514 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:46.256805 21514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:46.256984 21514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:46.257997 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "53f3f84a724540c1af1f0becba5d6a51"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "42012797d7238fd249280f6366f41126"
server_key_iv: "ba90b162bca4dd8574c052ed893022dc"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.258350 21514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "53f3f84a724540c1af1f0becba5d6a51"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "42012797d7238fd249280f6366f41126"
server_key_iv: "ba90b162bca4dd8574c052ed893022dc"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.259608 21514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:46.260283 21529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.260443 21514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:46.260535 21514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "53f3f84a724540c1af1f0becba5d6a51"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "42012797d7238fd249280f6366f41126"
server_key_iv: "ba90b162bca4dd8574c052ed893022dc"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.260622 21514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:46.278189 21514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:46.279929 21514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:46.280093 21514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:46.284139 21514 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:40607
I20260926 13:07:46.284189 21581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:40607 every 8 connection(s)
I20260926 13:07:46.284536 21514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:46.285427 21582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:46.287596 21582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: Bootstrap starting.
I20260926 13:07:46.287721 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21514
I20260926 13:07:46.287861 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:46.288057 2392 external_mini_cluster.cc:1428] Setting key 682b0dbdfd09a5f8630225494cde3b0c
I20260926 13:07:46.288347 21582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:46.288707 21582 log.cc:824] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:46.289448 21582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: No bootstrap required, opened a new log
I20260926 13:07:46.291724 21582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } }
I20260926 13:07:46.291914 21582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:46.291962 21582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f3f84a724540c1af1f0becba5d6a51, State: Initialized, Role: FOLLOWER
I20260926 13:07:46.292088 21582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } }
I20260926 13:07:46.292171 21582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:46.292213 21582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:46.292261 21582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:46.292908 21582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } }
I20260926 13:07:46.293011 21582 leader_election.cc:302] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f3f84a724540c1af1f0becba5d6a51; no voters:
I20260926 13:07:46.293169 21582 leader_election.cc:288] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:46.293195 21587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:46.293417 21582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:46.293402 21587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [term 1 LEADER]: Becoming Leader. State: Replica: 53f3f84a724540c1af1f0becba5d6a51, State: Running, Role: LEADER
I20260926 13:07:46.293613 21587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } }
I20260926 13:07:46.293939 21587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f3f84a724540c1af1f0becba5d6a51. Latest consensus state: current_term: 1 leader_uuid: "53f3f84a724540c1af1f0becba5d6a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } } }
I20260926 13:07:46.294080 21587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:46.294096 21589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f3f84a724540c1af1f0becba5d6a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f3f84a724540c1af1f0becba5d6a51" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 40607 } } }
I20260926 13:07:46.294253 21589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:46.294405 21596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:46.294896 21596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:46.296589 21596 catalog_manager.cc:1347] Generated new cluster ID: f973a010b95149e89c685ae0b5e5a410
I20260926 13:07:46.296653 21596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:46.299924 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.288576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.2.86.62:40607)
0926 13:07:46.288757 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.288767 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.288802 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:07:46.289405 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.289413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.289472 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.289624 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:46.290212 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.291679 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.292585 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.293249 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.293357 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.296463 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:46.296466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:46.296466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:46.296479 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:46.298006 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.298388 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.298391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.298391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.298491 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.298696 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.298697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.298697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.298974 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:46.299097 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.299603 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.299738 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260926 13:07:46.320818 21596 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:46.321403 21596 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:46.326215 21596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: Generated new TSK 0
I20260926 13:07:46.326421 21596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.86.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
Entry for principal kudu/127.2.86.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab.
I20260926 13:07:46.356096 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:46.436309 21608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:46.436501 21608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:46.436523 21608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:46.437712 21608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:46.437754 21608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:46.437806 21608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:46.439225 21608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.1.keytab
--principal=kudu/127.2.86.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:46.439468 21608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:46.439700 21608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:46.444510 21616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.444501 21614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.444501 21613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:46.444809 21608 server_base.cc:1034] running on GCE node
I20260926 13:07:46.445039 21608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:46.445282 21608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:46.446457 21608 hybrid_clock.cc:648] HybridClock initialized: now 1790428066446445 us; error 28 us; skew 500 ppm
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:46.448799 21608 init.cc:377] Logged in from keytab as kudu/127.2.86.1@KRBTEST.COM (short username kudu)
I20260926 13:07:46.449242 21608 webserver.cc:460] Webserver started at http://127.2.86.1:36595/ using document root <none> and password file <none>
I20260926 13:07:46.449430 21608 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:46.449469 21608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:46.449553 21608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:46.450523 21608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "c3bead983fbd4683a18912d6c1cd4326"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "3a3424f5db8992635316efe092312325"
server_key_iv: "ae0f3c59fa894d15e55bbf1c52f9b53e"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.450803 21608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "c3bead983fbd4683a18912d6c1cd4326"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "3a3424f5db8992635316efe092312325"
server_key_iv: "ae0f3c59fa894d15e55bbf1c52f9b53e"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.451989 21608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:46.452641 21623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.452816 21608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:46.452931 21608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c3bead983fbd4683a18912d6c1cd4326"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "3a3424f5db8992635316efe092312325"
server_key_iv: "ae0f3c59fa894d15e55bbf1c52f9b53e"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.453016 21608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:46.474112 21608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:46.476056 21608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:46.476325 21608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:46.476580 21608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:46.476969 21608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:46.477022 21608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.477062 21608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:46.477092 21608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.482856 21608 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:33189
I20260926 13:07:46.482905 21736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:33189 every 8 connection(s)
I20260926 13:07:46.483206 21608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.1@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:46.490988 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21608
I20260926 13:07:46.491125 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:46.491386 2392 external_mini_cluster.cc:1428] Setting key 101e0edff1a3b849793cc5cab81b090f
I20260926 13:07:46.492291 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.483819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40607 (local address 127.2.86.1:33941)
0926 13:07:46.483979 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.484019 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.484486 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.484720 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.484723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.485287 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:46.485472 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.485477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.486115 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.486115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.486899 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.486901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.487234 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.487235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.487334 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.487746 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:46.487754 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:46.489142 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:46.490708 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.490715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.490717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.491029 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.491279 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.491279 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.491280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.491418 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.491742 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:46.491748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:46.491880 (+ 132us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.491997 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.492140 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260926 13:07:46.492367 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.483867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:33941 (local address 127.2.86.62:40607)
0926 13:07:46.483933 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.483935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.484105 (+ 170us) server_negotiation.cc:407] Connection header received
0926 13:07:46.484566 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.484567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.484577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.484636 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:46.485550 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.486031 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.486968 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.487166 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.487215 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.489247 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:46.489249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:46.489249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:46.489264 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:46.490609 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.491128 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.491130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.491130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.491192 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.491489 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.491490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.491490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.491612 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:46.491701 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.492144 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.492237 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 13:07:46.492769 21737 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40607
I20260926 13:07:46.492888 21737 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:46.493075 21737 heartbeater.cc:511] Master 127.2.86.62:40607 requested a full tablet report, sending...
I20260926 13:07:46.493618 21546 ts_manager.cc:194] Registered new tserver with Master: c3bead983fbd4683a18912d6c1cd4326 (127.2.86.1:33189)
I20260926 13:07:46.494591 21546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.1@KRBTEST.COM'} at 127.2.86.1:33941
WARNING: no policy specified for kudu/127.2.86.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
Entry for principal kudu/127.2.86.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab.
I20260926 13:07:46.514055 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:46.592417 21742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:46.592607 21742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:46.592630 21742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:46.593880 21742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:46.593922 21742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:46.593950 21742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:46.595561 21742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.2.keytab
--principal=kudu/127.2.86.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:46.595747 21742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:46.595923 21742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:46.600574 21750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.600574 21748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.600590 21747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:46.601008 21742 server_base.cc:1034] running on GCE node
I20260926 13:07:46.601173 21742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:46.601351 21742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:46.602484 21742 hybrid_clock.cc:648] HybridClock initialized: now 1790428066602465 us; error 33 us; skew 500 ppm
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:46.604786 21742 init.cc:377] Logged in from keytab as kudu/127.2.86.2@KRBTEST.COM (short username kudu)
I20260926 13:07:46.605207 21742 webserver.cc:460] Webserver started at http://127.2.86.2:36873/ using document root <none> and password file <none>
I20260926 13:07:46.605438 21742 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:46.605504 21742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:46.605651 21742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:46.606647 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "3c26e4ce86154354b9b96da098217135"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "39422f71ae72d02e674edbc637276def"
server_key_iv: "b3c64f397a1b476e3c5e53ce8df4b652"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.606992 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "3c26e4ce86154354b9b96da098217135"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "39422f71ae72d02e674edbc637276def"
server_key_iv: "b3c64f397a1b476e3c5e53ce8df4b652"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.608265 21742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:46.608956 21757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.609117 21742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:46.609211 21742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "3c26e4ce86154354b9b96da098217135"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "39422f71ae72d02e674edbc637276def"
server_key_iv: "b3c64f397a1b476e3c5e53ce8df4b652"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.609294 21742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:46.630595 21742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:46.633038 21742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:46.633208 21742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:46.633445 21742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:46.633770 21742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:46.633826 21742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.633862 21742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:46.633889 21742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.639525 21742 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:35021
I20260926 13:07:46.639597 21870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:35021 every 8 connection(s)
I20260926 13:07:46.639941 21742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.2@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:46.648465 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21742
I20260926 13:07:46.648598 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:46.648764 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.640712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:48139 (local address 127.2.86.62:40607)
0926 13:07:46.640780 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.640782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.641018 (+ 236us) server_negotiation.cc:407] Connection header received
0926 13:07:46.641611 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.641612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.641621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.641678 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:46.642564 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.643043 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.643986 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.644172 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.644222 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.646293 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:46.646294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:46.646295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:46.646301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:46.647483 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.647846 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.647847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.647847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.647893 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.648094 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.648095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.648095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.648191 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:46.648241 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.648585 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.648666 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:07:46.648831 2392 external_mini_cluster.cc:1428] Setting key 1368055b8458fa044d64f1ec1d0d47c5
I20260926 13:07:46.649048 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.640687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40607 (local address 127.2.86.2:48139)
0926 13:07:46.640848 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.640929 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.641514 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.641743 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.641747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.642301 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:46.642491 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.642496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.643122 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.643123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.643917 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.643918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.644240 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.644241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.644342 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.644791 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:46.644798 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:46.646198 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:46.647550 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.647556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.647558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.647784 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.647946 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.647947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.647947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.648041 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.648264 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:46.648269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:46.648445 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.648560 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.648733 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260926 13:07:46.649483 21871 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40607
I20260926 13:07:46.649578 21871 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:46.649755 21871 heartbeater.cc:511] Master 127.2.86.62:40607 requested a full tablet report, sending...
I20260926 13:07:46.650159 21546 ts_manager.cc:194] Registered new tserver with Master: 3c26e4ce86154354b9b96da098217135 (127.2.86.2:35021)
I20260926 13:07:46.650525 21546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.2@KRBTEST.COM'} at 127.2.86.2:48139
WARNING: no policy specified for kudu/127.2.86.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
Entry for principal kudu/127.2.86.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab.
I20260926 13:07:46.673240 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:46.752468 21876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:46.752657 21876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:46.752679 21876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:46.753805 21876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:46.753844 21876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:46.753871 21876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:46.755277 21876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.3.keytab
--principal=kudu/127.2.86.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:40607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:46.755506 21876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:46.755769 21876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:46.760671 21882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.760707 21884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:46.760668 21881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:46.761018 21876 server_base.cc:1034] running on GCE node
I20260926 13:07:46.761270 21876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:46.761502 21876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:46.762642 21876 hybrid_clock.cc:648] HybridClock initialized: now 1790428066762629 us; error 29 us; skew 500 ppm
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:46.765184 21876 init.cc:377] Logged in from keytab as kudu/127.2.86.3@KRBTEST.COM (short username kudu)
I20260926 13:07:46.765687 21876 webserver.cc:460] Webserver started at http://127.2.86.3:36491/ using document root <none> and password file <none>
I20260926 13:07:46.765906 21876 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:46.765973 21876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:46.766103 21876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:46.767123 21876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "6e48c1712cc24abb8e899e89eb03915c"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "1a4ea09382ce1265ab246d991875b56b"
server_key_iv: "0bffbf9bf81a17a0e88297b9c6f61a70"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.767501 21876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "6e48c1712cc24abb8e899e89eb03915c"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "1a4ea09382ce1265ab246d991875b56b"
server_key_iv: "0bffbf9bf81a17a0e88297b9c6f61a70"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.768946 21876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:46.769722 21891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.769912 21876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:46.769999 21876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "6e48c1712cc24abb8e899e89eb03915c"
format_stamp: "Formatted at 2026-09-26 13:07:46 on dist-test-slave-j4lq"
server_key: "1a4ea09382ce1265ab246d991875b56b"
server_key_iv: "0bffbf9bf81a17a0e88297b9c6f61a70"
server_key_version: "encryptionkey@0"
I20260926 13:07:46.770051 21876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:46.793267 21876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:46.795908 21876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:46.796030 21876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:46.796218 21876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:46.796466 21876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:46.796495 21876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.796517 21876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:46.796530 21876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:46.802372 21876 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:43463
I20260926 13:07:46.802412 22004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:43463 every 8 connection(s)
I20260926 13:07:46.802754 21876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:46.808053 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 21876
I20260926 13:07:46.808137 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:46.808269 2392 external_mini_cluster.cc:1428] Setting key 30648ab9a8e4384f810e47b3325f9f41
Sep 26 13:07:46 dist-test-slave-j4lq krb5kdc[21501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428066, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.3@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:07:46.811797 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.803502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:54289 (local address 127.2.86.62:40607)
0926 13:07:46.803558 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.803560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.803722 (+ 162us) server_negotiation.cc:407] Connection header received
0926 13:07:46.804225 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.804226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.804236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.804290 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:46.805239 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.805799 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.806739 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.806971 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.807031 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.809241 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:46.809242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:46.809242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:46.809249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:46.810671 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.811005 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.811006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.811006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.811054 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.811299 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.811300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.811300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.811396 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:46.811444 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.811676 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.811735 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 13:07:46.811968 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.803477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:40607 (local address 127.2.86.3:54289)
0926 13:07:46.803624 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.803656 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.804111 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.804341 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.804345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.804961 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:46.805158 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.805164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.805876 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.805877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.806658 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.806660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.807047 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.807048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.807155 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.807614 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:07:46.807621 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:46.809158 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:46.810737 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.810742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.810744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.810941 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.811126 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:07:46.811127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:07:46.811127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:07:46.811245 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:07:46.811447 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:46.811451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:46.811560 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.811648 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.811759 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260926 13:07:46.812458 22005 heartbeater.cc:347] Connected to a master server at 127.2.86.62:40607
I20260926 13:07:46.812566 22005 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:46.812762 22005 heartbeater.cc:511] Master 127.2.86.62:40607 requested a full tablet report, sending...
I20260926 13:07:46.813225 21546 ts_manager.cc:194] Registered new tserver with Master: 6e48c1712cc24abb8e899e89eb03915c (127.2.86.3:43463)
I20260926 13:07:46.813592 21546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.86.3@KRBTEST.COM'} at 127.2.86.3:54289
I20260926 13:07:46.819973 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:46.827143 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.821259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.2.86.62:40607)
0926 13:07:46.821321 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.821323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.821452 (+ 129us) server_negotiation.cc:407] Connection header received
0926 13:07:46.821537 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.821538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.821548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.821611 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:46.822143 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.822626 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.823352 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.823557 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.823643 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.824740 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:46.824741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:46.824742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:46.824749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:46.825913 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.826304 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.826306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.826307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.826380 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:46.826586 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:46.826587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:46.826587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:46.826757 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:46.826866 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.826964 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.827042 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:07:46.828605 21546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:46.830286 21546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:46.836584 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.832550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35021 (local address 127.0.0.1:50384)
0926 13:07:46.832808 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.832846 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.832947 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.833139 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.833142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.833144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:46.833209 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.833211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834281 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.834283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.835510 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.835513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836265 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.836342 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836356 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.836458 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.836513 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":56,"threads_started":1}
I20260926 13:07:46.836596 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.832801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.2.86.2:35021)
0926 13:07:46.832974 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.832978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.832989 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:46.833028 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.833030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.833044 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.833135 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:46.833368 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834188 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.835595 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836179 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836328 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.836477 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.836513 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":118,"threads_started":1}
I20260926 13:07:46.836894 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.832556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:43463 (local address 127.0.0.1:47190)
0926 13:07:46.832809 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.832848 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.832946 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.833268 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.833270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.833272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:46.833328 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.833331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834641 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.834641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.835667 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.835668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836622 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.836702 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836706 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.836759 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.836797 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":21,"thread_start_us":103,"threads_started":1}
I20260926 13:07:46.836966 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.832641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.2.86.3:43463)
0926 13:07:46.832785 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.832790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.833039 (+ 249us) server_negotiation.cc:407] Connection header received
0926 13:07:46.833088 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.833091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.833108 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.833179 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:46.833410 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834534 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.835745 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836459 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836530 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836584 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.836783 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.836837 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260926 13:07:46.837131 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.832563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33189 (local address 127.0.0.1:35272)
0926 13:07:46.833232 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.833242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.833284 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.833557 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.833558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.833561 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:46.833602 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.833604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834802 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.834803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.835742 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.835744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836624 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.836959 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836964 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.837021 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.837064 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":2,"spinlock_wait_cycles":896,"thread_start_us":105,"threads_started":1}
I20260926 13:07:46.837395 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.833039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.2.86.1:33189)
0926 13:07:46.833399 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.833404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.833415 (+ 11us) server_negotiation.cc:407] Connection header received
0926 13:07:46.833453 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.833455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.833469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.833547 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:46.833675 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.834700 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.835815 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.836466 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.836530 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.836593 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.837272 (+ 679us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.837319 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":81,"threads_started":1}
I20260926 13:07:46.837553 21805 tablet_service.cc:1467] Processing CreateTablet for tablet b7c5baae97d64156ab34184de36c005e (DEFAULT_TABLE table=test-table [id=ee82e72b2d8f4e5fb2a2329cec0d5004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:46.837644 21939 tablet_service.cc:1467] Processing CreateTablet for tablet b7c5baae97d64156ab34184de36c005e (DEFAULT_TABLE table=test-table [id=ee82e72b2d8f4e5fb2a2329cec0d5004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:46.837929 21939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c5baae97d64156ab34184de36c005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:46.837937 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c5baae97d64156ab34184de36c005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:46.838012 21671 tablet_service.cc:1467] Processing CreateTablet for tablet b7c5baae97d64156ab34184de36c005e (DEFAULT_TABLE table=test-table [id=ee82e72b2d8f4e5fb2a2329cec0d5004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:46.838292 21671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c5baae97d64156ab34184de36c005e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:46.840255 22022 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Bootstrap starting.
I20260926 13:07:46.840852 22022 tablet_bootstrap.cc:654] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:46.841471 22022 log.cc:824] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:46.841658 22024 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Bootstrap starting.
I20260926 13:07:46.841820 22023 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Bootstrap starting.
I20260926 13:07:46.842240 22022 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: No bootstrap required, opened a new log
I20260926 13:07:46.842330 22022 ts_tablet_manager.cc:1397] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 13:07:46.842329 22024 tablet_bootstrap.cc:654] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:46.842392 22023 tablet_bootstrap.cc:654] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:46.842952 22024 log.cc:824] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:46.842983 22023 log.cc:824] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:46.843693 22023 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: No bootstrap required, opened a new log
I20260926 13:07:46.843762 22023 ts_tablet_manager.cc:1397] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:46.844048 22024 tablet_bootstrap.cc:492] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: No bootstrap required, opened a new log
I20260926 13:07:46.844128 22024 ts_tablet_manager.cc:1397] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 13:07:46.844190 22022 raft_consensus.cc:348] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.844373 22022 raft_consensus.cc:374] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:46.844417 22022 raft_consensus.cc:729] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e48c1712cc24abb8e899e89eb03915c, State: Initialized, Role: FOLLOWER
I20260926 13:07:46.844548 22022 consensus_queue.cc:260] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.844822 22022 ts_tablet_manager.cc:1428] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 13:07:46.844970 22005 heartbeater.cc:503] Master 127.2.86.62:40607 was elected leader, sending a full tablet report...
I20260926 13:07:46.845849 22023 raft_consensus.cc:348] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.846028 22023 raft_consensus.cc:374] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:46.846067 22023 raft_consensus.cc:729] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c26e4ce86154354b9b96da098217135, State: Initialized, Role: FOLLOWER
I20260926 13:07:46.846052 22024 raft_consensus.cc:348] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.846176 22023 consensus_queue.cc:260] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.846213 22024 raft_consensus.cc:374] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:46.846246 22024 raft_consensus.cc:729] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bead983fbd4683a18912d6c1cd4326, State: Initialized, Role: FOLLOWER
I20260926 13:07:46.846355 22024 consensus_queue.cc:260] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.846421 22023 ts_tablet_manager.cc:1428] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:46.846496 21871 heartbeater.cc:503] Master 127.2.86.62:40607 was elected leader, sending a full tablet report...
I20260926 13:07:46.846603 22024 ts_tablet_manager.cc:1428] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 13:07:46.846650 21737 heartbeater.cc:503] Master 127.2.86.62:40607 was elected leader, sending a full tablet report...
W20260926 13:07:46.890547 21872 tablet.cc:2369] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 13:07:46.983880 21738 tablet.cc:2369] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:07:46.984246 22028 raft_consensus.cc:482] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:46.984349 22028 raft_consensus.cc:504] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.984654 22028 leader_election.cc:288] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c26e4ce86154354b9b96da098217135 (127.2.86.2:35021), c3bead983fbd4683a18912d6c1cd4326 (127.2.86.1:33189)
I20260926 13:07:46.987856 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.984940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:35485 (local address 127.2.86.2:35021)
0926 13:07:46.984983 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.984985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.985178 (+ 193us) server_negotiation.cc:407] Connection header received
0926 13:07:46.985255 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.985256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.985270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.985331 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:46.985502 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.985968 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987138 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.987496 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987582 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.987592 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.987736 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.987779 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:07:46.987867 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.984860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33189 (local address 127.2.86.3:60919)
0926 13:07:46.984954 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.984969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.985013 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.985239 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.985240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.985242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:46.985320 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.985323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.985992 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.985993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.986982 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.986985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.987588 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.987669 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.987674 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.987751 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.987789 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":201}
I20260926 13:07:46.987879 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.984939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:60919 (local address 127.2.86.1:33189)
0926 13:07:46.984983 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:46.984985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:46.984998 (+ 13us) server_negotiation.cc:407] Connection header received
0926 13:07:46.985089 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:46.985090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:46.985104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:46.985186 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:46.985401 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.985906 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987096 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.987493 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987582 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.987593 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:46.987774 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 13:07:46.987815 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:07:46.987879 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:46.984860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:35021 (local address 127.2.86.3:35485)
0926 13:07:46.985086 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:46.985097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:46.985170 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:46.985342 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:46.985342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:46.985344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:46.985433 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.985435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.986057 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.986058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.987003 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:46.987006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:46.987584 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:46.987585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:46.987665 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:46.987669 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:46.987712 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 13:07:46.987773 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":21,"thread_start_us":116,"threads_started":1}
I20260926 13:07:46.988265 21691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c5baae97d64156ab34184de36c005e" candidate_uuid: "6e48c1712cc24abb8e899e89eb03915c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bead983fbd4683a18912d6c1cd4326" is_pre_election: true
I20260926 13:07:46.988440 21691 raft_consensus.cc:2454] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e48c1712cc24abb8e899e89eb03915c in term 0.
I20260926 13:07:46.988502 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c5baae97d64156ab34184de36c005e" candidate_uuid: "6e48c1712cc24abb8e899e89eb03915c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c26e4ce86154354b9b96da098217135" is_pre_election: true
I20260926 13:07:46.988684 21825 raft_consensus.cc:2454] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e48c1712cc24abb8e899e89eb03915c in term 0.
I20260926 13:07:46.988668 21894 leader_election.cc:302] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e48c1712cc24abb8e899e89eb03915c, c3bead983fbd4683a18912d6c1cd4326; no voters:
I20260926 13:07:46.988818 22028 raft_consensus.cc:2781] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:46.988906 22028 raft_consensus.cc:482] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:46.988936 22028 raft_consensus.cc:3037] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:46.989665 22028 raft_consensus.cc:504] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.989883 22028 leader_election.cc:288] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [CANDIDATE]: Term 1 election: Requested vote from peers 3c26e4ce86154354b9b96da098217135 (127.2.86.2:35021), c3bead983fbd4683a18912d6c1cd4326 (127.2.86.1:33189)
I20260926 13:07:46.989969 21691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c5baae97d64156ab34184de36c005e" candidate_uuid: "6e48c1712cc24abb8e899e89eb03915c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bead983fbd4683a18912d6c1cd4326"
I20260926 13:07:46.990024 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c5baae97d64156ab34184de36c005e" candidate_uuid: "6e48c1712cc24abb8e899e89eb03915c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c26e4ce86154354b9b96da098217135"
I20260926 13:07:46.990074 21691 raft_consensus.cc:3037] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:46.990136 21825 raft_consensus.cc:3037] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:46.990787 21691 raft_consensus.cc:2454] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e48c1712cc24abb8e899e89eb03915c in term 1.
I20260926 13:07:46.990954 21825 raft_consensus.cc:2454] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e48c1712cc24abb8e899e89eb03915c in term 1.
I20260926 13:07:46.990976 21894 leader_election.cc:302] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e48c1712cc24abb8e899e89eb03915c, c3bead983fbd4683a18912d6c1cd4326; no voters:
I20260926 13:07:46.991073 22028 raft_consensus.cc:2781] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:46.991236 22028 raft_consensus.cc:686] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 1 LEADER]: Becoming Leader. State: Replica: 6e48c1712cc24abb8e899e89eb03915c, State: Running, Role: LEADER
I20260926 13:07:46.991345 22028 consensus_queue.cc:237] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } }
I20260926 13:07:46.992203 21546 catalog_manager.cc:5495] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e48c1712cc24abb8e899e89eb03915c (127.2.86.3). New cstate: current_term: 1 leader_uuid: "6e48c1712cc24abb8e899e89eb03915c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e48c1712cc24abb8e899e89eb03915c" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 43463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20260926 13:07:47.020552 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.018507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.2.86.3:43463)
0926 13:07:47.018558 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.018560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.018567 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:07:47.018602 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.018603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.018621 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.018678 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:47.018789 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.019250 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.019947 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.020122 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.020177 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.020299 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:47.020407 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:47.020464 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.020478 (+ 14us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.020494 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:07:47.022426 21825 raft_consensus.cc:1261] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 1 FOLLOWER]: Refusing update from remote peer 6e48c1712cc24abb8e899e89eb03915c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:47.022431 21691 raft_consensus.cc:1261] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 1 FOLLOWER]: Refusing update from remote peer 6e48c1712cc24abb8e899e89eb03915c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:47.022787 22032 consensus_queue.cc:1035] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3bead983fbd4683a18912d6c1cd4326" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:47.022908 22028 consensus_queue.cc:1035] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c26e4ce86154354b9b96da098217135" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 35021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:47.025264 22035 mvcc.cc:205] Tried to move back new op lower bound from 7333593362521124864 to 7333593362397184000. Current Snapshot: MvccSnapshot[applied={T|T < 7333593362521124864}]
I20260926 13:07:47.025393 22036 mvcc.cc:205] Tried to move back new op lower bound from 7333593362521124864 to 7333593362397184000. Current Snapshot: MvccSnapshot[applied={T|T < 7333593362521124864}]
I20260926 13:07:47.027926 21545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:47.028038 21545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:47.028810 21545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53f3f84a724540c1af1f0becba5d6a51: Sending DeleteTablet for 3 replicas of tablet b7c5baae97d64156ab34184de36c005e
I20260926 13:07:47.029181 21939 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c5baae97d64156ab34184de36c005e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:47190
I20260926 13:07:47.029186 21805 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c5baae97d64156ab34184de36c005e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:50384
I20260926 13:07:47.029394 21671 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c5baae97d64156ab34184de36c005e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='kudu', principal='kudu/127.2.86.62@KRBTEST.COM'} at 127.0.0.1:35272
I20260926 13:07:47.029430 22045 tablet_replica.cc:330] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: stopping tablet replica
I20260926 13:07:47.029523 22045 raft_consensus.cc:2229] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:07:47.029575 22046 tablet_replica.cc:330] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: stopping tablet replica
I20260926 13:07:47.029624 22045 raft_consensus.cc:2258] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.029667 22046 raft_consensus.cc:2229] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:47.029770 22046 raft_consensus.cc:2258] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.029783 22044 tablet_replica.cc:330] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: stopping tablet replica
I20260926 13:07:47.029871 22044 raft_consensus.cc:2229] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:47.029961 22044 raft_consensus.cc:2258] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.030185 22046 ts_tablet_manager.cc:1905] T b7c5baae97d64156ab34184de36c005e P c3bead983fbd4683a18912d6c1cd4326: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:47.030191 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21608
I20260926 13:07:47.033897 22045 ts_tablet_manager.cc:1905] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:47.033933 22044 ts_tablet_manager.cc:1905] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:47.035964 22044 ts_tablet_manager.cc:1918] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:47.035964 22045 ts_tablet_manager.cc:1918] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:47.036043 22044 log.cc:1196] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/b7c5baae97d64156ab34184de36c005e
I20260926 13:07:47.036141 22045 log.cc:1196] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/b7c5baae97d64156ab34184de36c005e
I20260926 13:07:47.036366 22044 ts_tablet_manager.cc:1939] T b7c5baae97d64156ab34184de36c005e P 3c26e4ce86154354b9b96da098217135: Deleting consensus metadata
I20260926 13:07:47.036518 22045 ts_tablet_manager.cc:1939] T b7c5baae97d64156ab34184de36c005e P 6e48c1712cc24abb8e899e89eb03915c: Deleting consensus metadata
I20260926 13:07:47.036818 21530 catalog_manager.cc:4841] TS 3c26e4ce86154354b9b96da098217135 (127.2.86.2:35021): tablet b7c5baae97d64156ab34184de36c005e (table test-table [id=ee82e72b2d8f4e5fb2a2329cec0d5004]) successfully deleted
I20260926 13:07:47.037070 21531 catalog_manager.cc:4841] TS 6e48c1712cc24abb8e899e89eb03915c (127.2.86.3:43463): tablet b7c5baae97d64156ab34184de36c005e (table test-table [id=ee82e72b2d8f4e5fb2a2329cec0d5004]) successfully deleted
W20260926 13:07:47.037968 21532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:33189 (error 108)
I20260926 13:07:47.038203 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21742
I20260926 13:07:47.041770 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.041623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:33189 (local address 127.0.0.1:35288)
0926 13:07:47.041689 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.041735 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 13:07:47.041953 21532 catalog_manager.cc:4584] TS c3bead983fbd4683a18912d6c1cd4326 (127.2.86.1:33189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7c5baae97d64156ab34184de36c005e: Network error: Client connection negotiation failed: client connection to 127.2.86.1:33189: connect: Connection refused (error 111)
I20260926 13:07:47.043414 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21876
I20260926 13:07:47.047909 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 21514
2026-09-26T13:07:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2023 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T13:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:47Z Disabled control of system clock
I20260926 13:07:47.074944 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:32883
--webserver_interface=127.2.86.62
--webserver_port=0
--builtin_ntp_servers=127.2.86.20:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:32883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:47.153555 22051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:47.153787 22051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:47.153808 22051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:47.154966 22051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:47.155006 22051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:47.155019 22051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:47.155031 22051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:47.155090 22051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:47.156459 22051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:32883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:32883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:47.156697 22051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:47.157039 22051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:47.161685 22056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.161712 22057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.161685 22059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:47.161914 22051 server_base.cc:1034] running on GCE node
I20260926 13:07:47.162160 22051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:47.162468 22051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:47.163631 22051 hybrid_clock.cc:648] HybridClock initialized: now 1790428067163619 us; error 42 us; skew 500 ppm
I20260926 13:07:47.164829 22051 webserver.cc:460] Webserver started at http://127.2.86.62:44781/ using document root <none> and password file <none>
I20260926 13:07:47.165103 22051 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:47.165174 22051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:47.165325 22051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:47.166373 22051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "c711071f544c42bb809388ec573d3653"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "271c79dfc387b91b69e5d2518926c37c"
server_key_iv: "023e8633966d8491f2101806fa4e5933"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.166738 22051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "c711071f544c42bb809388ec573d3653"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "271c79dfc387b91b69e5d2518926c37c"
server_key_iv: "023e8633966d8491f2101806fa4e5933"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.167992 22051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:47.168726 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.168916 22051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:47.169014 22051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "c711071f544c42bb809388ec573d3653"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "271c79dfc387b91b69e5d2518926c37c"
server_key_iv: "023e8633966d8491f2101806fa4e5933"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.169102 22051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:47.185766 22051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:47.186053 22051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:47.186219 22051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:47.190090 22051 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:32883
I20260926 13:07:47.190129 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:32883 every 8 connection(s)
I20260926 13:07:47.190450 22051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:47.191351 22118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:47.193543 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: Bootstrap starting.
I20260926 13:07:47.194077 22118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:47.194319 22118 log.cc:824] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:47.194900 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: No bootstrap required, opened a new log
I20260926 13:07:47.196386 22118 raft_consensus.cc:348] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } }
I20260926 13:07:47.196512 22118 raft_consensus.cc:374] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:47.196534 22118 raft_consensus.cc:729] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c711071f544c42bb809388ec573d3653, State: Initialized, Role: FOLLOWER
I20260926 13:07:47.196668 22118 consensus_queue.cc:260] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } }
I20260926 13:07:47.196748 22118 raft_consensus.cc:388] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:47.196780 22118 raft_consensus.cc:482] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:47.196813 22118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:47.197343 22118 raft_consensus.cc:504] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } }
I20260926 13:07:47.197423 22118 leader_election.cc:302] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c711071f544c42bb809388ec573d3653; no voters:
I20260926 13:07:47.197538 22118 leader_election.cc:288] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:47.197621 22121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:47.197784 22121 raft_consensus.cc:686] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [term 1 LEADER]: Becoming Leader. State: Replica: c711071f544c42bb809388ec573d3653, State: Running, Role: LEADER
I20260926 13:07:47.197789 22118 sys_catalog.cc:564] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:47.197893 22121 consensus_queue.cc:237] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } }
I20260926 13:07:47.198259 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c711071f544c42bb809388ec573d3653. Latest consensus state: current_term: 1 leader_uuid: "c711071f544c42bb809388ec573d3653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } } }
I20260926 13:07:47.198354 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:47.198339 22122 sys_catalog.cc:455] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c711071f544c42bb809388ec573d3653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c711071f544c42bb809388ec573d3653" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 32883 } } }
I20260926 13:07:47.198422 22122 sys_catalog.cc:458] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:47.198567 22130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:47.199220 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22051
I20260926 13:07:47.199292 22130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:47.199339 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:47.199556 2392 external_mini_cluster.cc:1428] Setting key 0d3653f5e9ad933143cff87ba30ce956
I20260926 13:07:47.201339 22130 catalog_manager.cc:1347] Generated new cluster ID: 195f7cab413e4743aae7c8126b2449a3
I20260926 13:07:47.201392 22130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:47.204597 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.200025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.2.86.62:32883)
0926 13:07:47.200256 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.200265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.200294 (+ 29us) server_negotiation.cc:407] Connection header received
0926 13:07:47.200403 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.200409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.200472 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.200620 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:47.200897 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.201866 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.202671 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.203095 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.203226 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.203918 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:47.203922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:47.203922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:47.204095 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:47.204128 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:47.204139 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:47.204212 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.204291 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.204452 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
I20260926 13:07:47.205776 22130 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:47.206266 22130 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:47.213621 22130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: Generated new TSK 0
I20260926 13:07:47.213789 22130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:47.221253 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.1:0
--local_ip_for_outbound_sockets=127.2.86.1
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--builtin_ntp_servers=127.2.86.20:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 13:07:47.301355 22142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:47.301587 22142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:47.301625 22142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:47.302752 22142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:47.302810 22142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:47.302850 22142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.1
I20260926 13:07:47.304184 22142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.86.1
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.1
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:47.304435 22142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:47.304718 22142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:47.310473 22147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.310484 22148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.310580 22150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:47.310847 22142 server_base.cc:1034] running on GCE node
I20260926 13:07:47.311058 22142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:47.311316 22142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:47.312495 22142 hybrid_clock.cc:648] HybridClock initialized: now 1790428067312481 us; error 30 us; skew 500 ppm
I20260926 13:07:47.313891 22142 webserver.cc:460] Webserver started at http://127.2.86.1:38705/ using document root <none> and password file <none>
I20260926 13:07:47.314146 22142 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:47.314227 22142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:47.314378 22142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:47.315428 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data/instance:
uuid: "c96fb0fbf59d44bcab77d5030d2cd497"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "1a538ff14a39eec016efe8a36aa9e6b7"
server_key_iv: "972caf7c0f6b8055bf9d77b1b485bfbd"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.315795 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance:
uuid: "c96fb0fbf59d44bcab77d5030d2cd497"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "1a538ff14a39eec016efe8a36aa9e6b7"
server_key_iv: "972caf7c0f6b8055bf9d77b1b485bfbd"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.317184 22142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:47.317934 22156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.318126 22142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:47.318228 22142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal
uuid: "c96fb0fbf59d44bcab77d5030d2cd497"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "1a538ff14a39eec016efe8a36aa9e6b7"
server_key_iv: "972caf7c0f6b8055bf9d77b1b485bfbd"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.318308 22142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:47.328236 22142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:47.328547 22142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:47.328699 22142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:47.328994 22142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:47.329308 22142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:47.329370 22142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.329409 22142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:47.329437 22142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.335058 22142 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.1:37733
I20260926 13:07:47.335099 22269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.1:37733 every 8 connection(s)
I20260926 13:07:47.335429 22142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/data/info.pb
I20260926 13:07:47.336506 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22142
I20260926 13:07:47.336622 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-0/wal/instance
I20260926 13:07:47.336838 2392 external_mini_cluster.cc:1428] Setting key 3079a5db6013c4ea3cc5c2894083cc9d
I20260926 13:07:47.337929 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.2:0
--local_ip_for_outbound_sockets=127.2.86.2
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--builtin_ntp_servers=127.2.86.20:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:07:47.341449 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.336138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.1:57473 (local address 127.2.86.62:32883)
0926 13:07:47.336207 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.336209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.336431 (+ 222us) server_negotiation.cc:407] Connection header received
0926 13:07:47.337202 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.337203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.337218 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.337264 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:47.338287 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.338787 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.339877 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.340120 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.340202 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.341161 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:47.341162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:47.341162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:47.341167 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:47.341182 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:47.341186 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:47.341228 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.341336 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.341363 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 13:07:47.341655 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.336095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32883 (local address 127.2.86.1:57473)
0926 13:07:47.336291 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.336333 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.337068 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.337333 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.337337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.337346 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:47.338005 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.338012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.338900 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.338901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.339796 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.339798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.340211 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.340213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.340349 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.341003 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:47.341003 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:47.341032 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:47.341056 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:47.341240 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:47.341244 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:47.341247 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.341326 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.341428 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260926 13:07:47.342172 22270 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32883
I20260926 13:07:47.342278 22270 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:47.342460 22270 heartbeater.cc:511] Master 127.2.86.62:32883 requested a full tablet report, sending...
I20260926 13:07:47.342980 22082 ts_manager.cc:194] Registered new tserver with Master: c96fb0fbf59d44bcab77d5030d2cd497 (127.2.86.1:37733)
I20260926 13:07:47.343582 22082 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.1:57473
W20260926 13:07:47.417776 22273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:47.417973 22273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:47.417994 22273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:47.419144 22273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:47.419183 22273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:47.419235 22273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.2
I20260926 13:07:47.420617 22273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.86.2
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.2
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:47.420854 22273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:47.421213 22273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:47.426050 22278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.426065 22279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.426074 22281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:47.426280 22273 server_base.cc:1034] running on GCE node
I20260926 13:07:47.426505 22273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:47.426738 22273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:47.427917 22273 hybrid_clock.cc:648] HybridClock initialized: now 1790428067427910 us; error 34 us; skew 500 ppm
I20260926 13:07:47.429243 22273 webserver.cc:460] Webserver started at http://127.2.86.2:34371/ using document root <none> and password file <none>
I20260926 13:07:47.429491 22273 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:47.429562 22273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:47.429702 22273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:47.430706 22273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data/instance:
uuid: "df0538e33f5b4a349fe8f4ee11c43a10"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "8b865addde5d47651a679f7d260d7684"
server_key_iv: "8427d915e6859969d5e795d1e3ed570b"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.431053 22273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance:
uuid: "df0538e33f5b4a349fe8f4ee11c43a10"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "8b865addde5d47651a679f7d260d7684"
server_key_iv: "8427d915e6859969d5e795d1e3ed570b"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.432331 22273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:47.433049 22287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.433228 22273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:47.433321 22273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal
uuid: "df0538e33f5b4a349fe8f4ee11c43a10"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "8b865addde5d47651a679f7d260d7684"
server_key_iv: "8427d915e6859969d5e795d1e3ed570b"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.433404 22273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:47.441726 22273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:47.442014 22273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:47.442162 22273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:47.442384 22273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:47.442703 22273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:47.442755 22273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.442790 22273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:47.442816 22273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.448622 22273 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.2:34177
I20260926 13:07:47.448668 22400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.2:34177 every 8 connection(s)
I20260926 13:07:47.449003 22273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/data/info.pb
I20260926 13:07:47.452489 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22273
I20260926 13:07:47.452605 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal/instance
I20260926 13:07:47.452813 2392 external_mini_cluster.cc:1428] Setting key a1ac70f7f4776d4f304db5570c275cae
I20260926 13:07:47.453866 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.86.3:0
--local_ip_for_outbound_sockets=127.2.86.3
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--builtin_ntp_servers=127.2.86.20:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 13:07:47.454609 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.449675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.2:38853 (local address 127.2.86.62:32883)
0926 13:07:47.449738 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.449741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.449941 (+ 200us) server_negotiation.cc:407] Connection header received
0926 13:07:47.450409 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.450410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.450422 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.450500 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:47.451199 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.451696 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.452866 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.453161 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.453230 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.454312 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:47.454313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:47.454314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:47.454319 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:47.454341 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:47.454346 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:47.454406 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.454466 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.454493 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:07:47.454726 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.449639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32883 (local address 127.2.86.2:38853)
0926 13:07:47.449811 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.449851 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.450311 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.450550 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.450553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.450560 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:47.451027 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.451031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.451777 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.451777 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.452759 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.452762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.453252 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.453254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.453402 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.454130 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:47.454131 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:47.454171 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:47.454192 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:47.454412 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:47.454416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:47.454418 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.454467 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.454541 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260926 13:07:47.455118 22401 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32883
I20260926 13:07:47.455215 22401 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:47.455415 22401 heartbeater.cc:511] Master 127.2.86.62:32883 requested a full tablet report, sending...
I20260926 13:07:47.455745 22082 ts_manager.cc:194] Registered new tserver with Master: df0538e33f5b4a349fe8f4ee11c43a10 (127.2.86.2:34177)
I20260926 13:07:47.456125 22082 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.2:38853
W20260926 13:07:47.533066 22404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:47.533249 22404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:47.533270 22404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:47.534333 22404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:47.534365 22404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:47.534390 22404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.86.3
I20260926 13:07:47.535621 22404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.86.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.86.3
--webserver_port=0
--tserver_master_addrs=127.2.86.62:32883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.86.3
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:47.535789 22404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:47.536125 22404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:47.540953 22409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:47.540978 22404 server_base.cc:1034] running on GCE node
W20260926 13:07:47.540982 22412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:47.540954 22410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:47.541414 22404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:47.541672 22404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:47.542807 22404 hybrid_clock.cc:648] HybridClock initialized: now 1790428067542799 us; error 26 us; skew 500 ppm
I20260926 13:07:47.543968 22404 webserver.cc:460] Webserver started at http://127.2.86.3:45857/ using document root <none> and password file <none>
I20260926 13:07:47.544193 22404 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:47.544236 22404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:47.544401 22404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:47.545493 22404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data/instance:
uuid: "58515146ef2a46be825c8876cba72473"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "b2feeb52645dae793dac2477c9a994c8"
server_key_iv: "522d2d7a785088525e0498628d75864d"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.545841 22404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance:
uuid: "58515146ef2a46be825c8876cba72473"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "b2feeb52645dae793dac2477c9a994c8"
server_key_iv: "522d2d7a785088525e0498628d75864d"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.547781 22404 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:07:47.549516 22418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.549714 22404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:47.549820 22404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal
uuid: "58515146ef2a46be825c8876cba72473"
format_stamp: "Formatted at 2026-09-26 13:07:47 on dist-test-slave-j4lq"
server_key: "b2feeb52645dae793dac2477c9a994c8"
server_key_iv: "522d2d7a785088525e0498628d75864d"
server_key_version: "encryptionkey@0"
I20260926 13:07:47.549901 22404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:47.560757 22404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:47.561038 22404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:47.561183 22404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:47.561386 22404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:07:47.561692 22404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:07:47.561746 22404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.561787 22404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:07:47.561813 22404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:47.567591 22404 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.3:39719
I20260926 13:07:47.567632 22531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.3:39719 every 8 connection(s)
I20260926 13:07:47.567962 22404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/data/info.pb
I20260926 13:07:47.568789 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22404
I20260926 13:07:47.568930 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal/instance
I20260926 13:07:47.569172 2392 external_mini_cluster.cc:1428] Setting key 98d4c1784e77845317860e5de383bee2
I20260926 13:07:47.574421 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.568682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:60829 (local address 127.2.86.62:32883)
0926 13:07:47.568732 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.568734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.569125 (+ 391us) server_negotiation.cc:407] Connection header received
0926 13:07:47.569898 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.569899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.569912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.569955 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:47.570837 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.571370 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.573020 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.573211 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.573265 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.574184 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:47.574186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:47.574186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:47.574190 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:47.574203 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:47.574207 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:47.574244 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.574341 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.574366 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:07:47.574709 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.568661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:32883 (local address 127.2.86.3:60829)
0926 13:07:47.568917 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.569003 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.569803 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.570036 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.570041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.570054 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:07:47.570749 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.570755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.571456 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.571457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.572939 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.572943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.573290 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.573292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.573457 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.573990 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 13:07:47.573991 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:07:47.574029 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 13:07:47.574058 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:07:47.574261 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:07:47.574266 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:07:47.574269 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.574340 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.574453 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260926 13:07:47.575199 22532 heartbeater.cc:347] Connected to a master server at 127.2.86.62:32883
I20260926 13:07:47.575297 22532 heartbeater.cc:464] Registering TS with master...
I20260926 13:07:47.575492 22532 heartbeater.cc:511] Master 127.2.86.62:32883 requested a full tablet report, sending...
I20260926 13:07:47.575809 22082 ts_manager.cc:194] Registered new tserver with Master: 58515146ef2a46be825c8876cba72473 (127.2.86.3:39719)
I20260926 13:07:47.576162 22082 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.86.3:60829
I20260926 13:07:47.580614 2392 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 13:07:47.585322 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.582075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.2.86.62:32883)
0926 13:07:47.582121 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.582123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.582226 (+ 103us) server_negotiation.cc:407] Connection header received
0926 13:07:47.582299 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.582299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.582308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.582349 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:47.582535 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.582968 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.584188 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.584356 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.584403 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.585111 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:47.585112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:47.585112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:47.585116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:47.585128 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:47.585131 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:47.585171 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.585236 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.585254 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:07:47.586949 22082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 13:07:47.588402 22082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 13:07:47.595243 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.3:39719 (local address 127.0.0.1:53534)
0926 13:07:47.591029 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.591081 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.591143 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.591805 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.591806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.591808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:47.591843 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.591844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.593050 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.593051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.594107 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.594109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.594964 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.594965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.595054 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.595068 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.595111 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.595165 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":60,"threads_started":1}
I20260926 13:07:47.595474 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.2.86.2:34177)
0926 13:07:47.591015 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.591020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.591131 (+ 111us) server_negotiation.cc:407] Connection header received
0926 13:07:47.591234 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.591236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.591251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.591338 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:47.591553 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.592644 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.594160 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.594866 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.594915 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.595005 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.595218 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.595255 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260926 13:07:47.595664 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.2.86.3:39719)
0926 13:07:47.591627 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.591632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.591642 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:07:47.591695 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.591697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.591711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.591791 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:47.591928 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.592960 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.594195 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.594866 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.595426 (+ 560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.595516 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.595544 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.595583 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":76,"threads_started":1}
I20260926 13:07:47.595685 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:34177 (local address 127.0.0.1:40246)
0926 13:07:47.591059 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.591076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.591146 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.591431 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.591433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.591435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:47.591478 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.591479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.592735 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.592736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.594029 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.594031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.595058 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.595058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.595117 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.595120 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.595579 (+ 459us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.595617 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":33,"thread_start_us":81,"threads_started":1}
I20260926 13:07:47.596262 22466 tablet_service.cc:1467] Processing CreateTablet for tablet e18005a87c9041e5bdff99ec029b416e (DEFAULT_TABLE table=test-table [id=57b878fa79924aa3ba88394518c5648d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:47.596444 22335 tablet_service.cc:1467] Processing CreateTablet for tablet e18005a87c9041e5bdff99ec029b416e (DEFAULT_TABLE table=test-table [id=57b878fa79924aa3ba88394518c5648d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:47.596585 22466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18005a87c9041e5bdff99ec029b416e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:47.596791 22335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18005a87c9041e5bdff99ec029b416e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:47.597241 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37733 (local address 127.0.0.1:35466)
0926 13:07:47.591245 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.591254 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.591296 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.592128 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.592129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.592131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:47.592183 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.592184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.593922 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.593923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.595278 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.595281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.596996 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.596997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.597060 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.597063 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.597128 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.597174 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":70,"threads_started":1}
I20260926 13:07:47.597467 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.590900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.2.86.1:37733)
0926 13:07:47.591097 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.591100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.591852 (+ 752us) server_negotiation.cc:407] Connection header received
0926 13:07:47.591953 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.591956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.591972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.592039 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:47.592268 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.593838 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.595944 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.596763 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.596834 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.596959 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.597343 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.597385 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260926 13:07:47.599547 22546 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Bootstrap starting.
I20260926 13:07:47.599853 22547 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Bootstrap starting.
I20260926 13:07:47.600229 22546 tablet_bootstrap.cc:654] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:47.600346 22204 tablet_service.cc:1467] Processing CreateTablet for tablet e18005a87c9041e5bdff99ec029b416e (DEFAULT_TABLE table=test-table [id=57b878fa79924aa3ba88394518c5648d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 13:07:47.600550 22546 log.cc:824] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:47.600693 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18005a87c9041e5bdff99ec029b416e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:47.600790 22547 tablet_bootstrap.cc:654] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:47.601219 22547 log.cc:824] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:47.601464 22546 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: No bootstrap required, opened a new log
I20260926 13:07:47.601531 22546 ts_tablet_manager.cc:1397] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 13:07:47.604065 22550 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: Bootstrap starting.
I20260926 13:07:47.604118 22546 raft_consensus.cc:348] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.604319 22546 raft_consensus.cc:374] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:47.604365 22546 raft_consensus.cc:729] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58515146ef2a46be825c8876cba72473, State: Initialized, Role: FOLLOWER
I20260926 13:07:47.604481 22546 consensus_queue.cc:260] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.604744 22546 ts_tablet_manager.cc:1428] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:47.604776 22550 tablet_bootstrap.cc:654] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:47.604910 22532 heartbeater.cc:503] Master 127.2.86.62:32883 was elected leader, sending a full tablet report...
I20260926 13:07:47.605439 22550 log.cc:824] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:47.609460 22547 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: No bootstrap required, opened a new log
I20260926 13:07:47.609535 22547 ts_tablet_manager.cc:1397] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260926 13:07:47.609712 22550 tablet_bootstrap.cc:492] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: No bootstrap required, opened a new log
I20260926 13:07:47.609798 22550 ts_tablet_manager.cc:1397] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 13:07:47.611871 22547 raft_consensus.cc:348] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.612044 22547 raft_consensus.cc:374] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:47.612079 22547 raft_consensus.cc:729] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df0538e33f5b4a349fe8f4ee11c43a10, State: Initialized, Role: FOLLOWER
I20260926 13:07:47.612210 22547 consensus_queue.cc:260] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.612295 22550 raft_consensus.cc:348] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.612452 22550 raft_consensus.cc:374] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:47.612481 22547 ts_tablet_manager.cc:1428] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:47.612497 22550 raft_consensus.cc:729] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96fb0fbf59d44bcab77d5030d2cd497, State: Initialized, Role: FOLLOWER
I20260926 13:07:47.612598 22401 heartbeater.cc:503] Master 127.2.86.62:32883 was elected leader, sending a full tablet report...
I20260926 13:07:47.612604 22550 consensus_queue.cc:260] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.612852 22550 ts_tablet_manager.cc:1428] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 13:07:47.612920 22270 heartbeater.cc:503] Master 127.2.86.62:32883 was elected leader, sending a full tablet report...
I20260926 13:07:47.620538 22551 raft_consensus.cc:482] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:07:47.620647 22551 raft_consensus.cc:504] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.621034 22551 leader_election.cc:288] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df0538e33f5b4a349fe8f4ee11c43a10 (127.2.86.2:34177), c96fb0fbf59d44bcab77d5030d2cd497 (127.2.86.1:37733)
I20260926 13:07:47.624017 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.621200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37733 (local address 127.2.86.3:40885)
0926 13:07:47.621245 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.621258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.621308 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.621577 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.621578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.621580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:47.621648 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.621649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.622365 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.622366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.623213 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.623215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.623754 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.623755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.623829 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.623832 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.623905 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.623947 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":4}
I20260926 13:07:47.624012 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.621390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:40885 (local address 127.2.86.1:37733)
0926 13:07:47.621437 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.621439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.621448 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:07:47.621471 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.621472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.621483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.621532 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:47.621736 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.622287 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.623304 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.623673 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.623755 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.623765 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.623914 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.623949 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 13:07:47.624413 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18005a87c9041e5bdff99ec029b416e" candidate_uuid: "58515146ef2a46be825c8876cba72473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" is_pre_election: true
I20260926 13:07:47.624550 22224 raft_consensus.cc:2454] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58515146ef2a46be825c8876cba72473 in term 0.
I20260926 13:07:47.624755 22421 leader_election.cc:302] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58515146ef2a46be825c8876cba72473, c96fb0fbf59d44bcab77d5030d2cd497; no voters:
I20260926 13:07:47.624889 22551 raft_consensus.cc:2781] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:07:47.624951 22551 raft_consensus.cc:482] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:07:47.624981 22551 raft_consensus.cc:3037] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:47.625283 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.621447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.86.3:56335 (local address 127.2.86.2:34177)
0926 13:07:47.621491 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.621493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.621501 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:07:47.621524 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.621524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.621535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.621594 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 13:07:47.621727 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.622566 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.624184 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.624850 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.624921 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.624930 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.625183 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.625228 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:07:47.625635 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.621200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.2:34177 (local address 127.2.86.3:56335)
0926 13:07:47.621392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.621402 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.621457 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.621591 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:07:47.621592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:07:47.621593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 13:07:47.621648 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.621649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.622689 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.622691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.624063 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:07:47.624066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.624997 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.624998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:07:47.625093 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.625097 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 13:07:47.625153 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 13:07:47.625201 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":11,"thread_start_us":81,"threads_started":1}
I20260926 13:07:47.625887 22551 raft_consensus.cc:504] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.625926 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18005a87c9041e5bdff99ec029b416e" candidate_uuid: "58515146ef2a46be825c8876cba72473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" is_pre_election: true
I20260926 13:07:47.626032 22551 leader_election.cc:288] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [CANDIDATE]: Term 1 election: Requested vote from peers df0538e33f5b4a349fe8f4ee11c43a10 (127.2.86.2:34177), c96fb0fbf59d44bcab77d5030d2cd497 (127.2.86.1:37733)
I20260926 13:07:47.626065 22355 raft_consensus.cc:2454] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58515146ef2a46be825c8876cba72473 in term 0.
I20260926 13:07:47.626209 22224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18005a87c9041e5bdff99ec029b416e" candidate_uuid: "58515146ef2a46be825c8876cba72473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96fb0fbf59d44bcab77d5030d2cd497"
I20260926 13:07:47.626286 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18005a87c9041e5bdff99ec029b416e" candidate_uuid: "58515146ef2a46be825c8876cba72473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df0538e33f5b4a349fe8f4ee11c43a10"
I20260926 13:07:47.626282 22224 raft_consensus.cc:3037] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:47.626346 22355 raft_consensus.cc:3037] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:47.627141 22224 raft_consensus.cc:2454] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58515146ef2a46be825c8876cba72473 in term 1.
I20260926 13:07:47.627141 22355 raft_consensus.cc:2454] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58515146ef2a46be825c8876cba72473 in term 1.
I20260926 13:07:47.627308 22421 leader_election.cc:302] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58515146ef2a46be825c8876cba72473, c96fb0fbf59d44bcab77d5030d2cd497; no voters:
I20260926 13:07:47.627388 22551 raft_consensus.cc:2781] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:47.627517 22551 raft_consensus.cc:686] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 1 LEADER]: Becoming Leader. State: Replica: 58515146ef2a46be825c8876cba72473, State: Running, Role: LEADER
I20260926 13:07:47.627605 22551 consensus_queue.cc:237] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } }
I20260926 13:07:47.628322 22081 catalog_manager.cc:5495] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58515146ef2a46be825c8876cba72473 (127.2.86.3). New cstate: current_term: 1 leader_uuid: "58515146ef2a46be825c8876cba72473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58515146ef2a46be825c8876cba72473" member_type: VOTER last_known_addr { host: "127.2.86.3" port: 39719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 } health_report { overall_health: UNKNOWN } } }
I20260926 13:07:47.639837 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:47.637435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.2.86.3:39719)
0926 13:07:47.637493 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:47.637496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:47.637504 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:07:47.637558 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:47.637559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:47.637573 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:47.637648 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:07:47.637794 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.638254 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.639025 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:47.639224 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:47.639301 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:47.639480 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:07:47.639575 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:07:47.639663 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:47.639716 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 13:07:47.639734 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 13:07:47.641542 22355 raft_consensus.cc:1261] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 1 FOLLOWER]: Refusing update from remote peer 58515146ef2a46be825c8876cba72473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:47.641551 22224 raft_consensus.cc:1261] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 1 FOLLOWER]: Refusing update from remote peer 58515146ef2a46be825c8876cba72473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:07:47.641881 22551 consensus_queue.cc:1035] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df0538e33f5b4a349fe8f4ee11c43a10" member_type: VOTER last_known_addr { host: "127.2.86.2" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:47.642007 22556 consensus_queue.cc:1035] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96fb0fbf59d44bcab77d5030d2cd497" member_type: VOTER last_known_addr { host: "127.2.86.1" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:07:47.644126 22559 mvcc.cc:205] Tried to move back new op lower bound from 7333593365057347584 to 7333593365003382784. Current Snapshot: MvccSnapshot[applied={T|T < 7333593365057347584}]
I20260926 13:07:47.644658 22560 mvcc.cc:205] Tried to move back new op lower bound from 7333593365057347584 to 7333593365003382784. Current Snapshot: MvccSnapshot[applied={T|T < 7333593365057347584}]
I20260926 13:07:47.644826 22561 mvcc.cc:205] Tried to move back new op lower bound from 7333593365057347584 to 7333593365003382784. Current Snapshot: MvccSnapshot[applied={T|T < 7333593365057347584}]
I20260926 13:07:47.646464 22082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:47.646567 22082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 13:07:47.647289 22082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c711071f544c42bb809388ec573d3653: Sending DeleteTablet for 3 replicas of tablet e18005a87c9041e5bdff99ec029b416e
I20260926 13:07:47.647599 22466 tablet_service.cc:1514] Processing DeleteTablet for tablet e18005a87c9041e5bdff99ec029b416e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='slave'} at 127.0.0.1:53534
I20260926 13:07:47.647622 22204 tablet_service.cc:1514] Processing DeleteTablet for tablet e18005a87c9041e5bdff99ec029b416e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='slave'} at 127.0.0.1:35466
I20260926 13:07:47.647642 22335 tablet_service.cc:1514] Processing DeleteTablet for tablet e18005a87c9041e5bdff99ec029b416e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 13:07:47 UTC) from {username='slave'} at 127.0.0.1:40246
I20260926 13:07:47.647811 22570 tablet_replica.cc:330] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497: stopping tablet replica
I20260926 13:07:47.647907 22570 raft_consensus.cc:2229] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:47.648012 22570 raft_consensus.cc:2258] T e18005a87c9041e5bdff99ec029b416e P c96fb0fbf59d44bcab77d5030d2cd497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.648175 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22142
I20260926 13:07:47.650017 22569 tablet_replica.cc:330] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: stopping tablet replica
I20260926 13:07:47.650122 22569 raft_consensus.cc:2229] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:07:47.650221 22569 raft_consensus.cc:2258] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.650360 22571 tablet_replica.cc:330] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: stopping tablet replica
I20260926 13:07:47.650458 22571 raft_consensus.cc:2229] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:07:47.650558 22571 raft_consensus.cc:2258] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:07:47.650977 22571 ts_tablet_manager.cc:1905] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 13:07:47.652956 22571 ts_tablet_manager.cc:1918] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:47.653033 22571 log.cc:1196] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-1/wal/wals/e18005a87c9041e5bdff99ec029b416e
I20260926 13:07:47.653290 22571 ts_tablet_manager.cc:1939] T e18005a87c9041e5bdff99ec029b416e P df0538e33f5b4a349fe8f4ee11c43a10: Deleting consensus metadata
I20260926 13:07:47.653755 22066 catalog_manager.cc:4841] TS df0538e33f5b4a349fe8f4ee11c43a10 (127.2.86.2:34177): tablet e18005a87c9041e5bdff99ec029b416e (table test-table [id=57b878fa79924aa3ba88394518c5648d]) successfully deleted
W20260926 13:07:47.654937 22068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.86.1:37733 (error 108)
I20260926 13:07:47.655679 22569 ts_tablet_manager.cc:1905] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 13:07:47.657039 22540 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:07:47.655325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.1:37733 (local address 127.0.0.1:35470)
0926 13:07:47.655383 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 13:07:47.655396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 13:07:47.655440 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:07:47.656984 (+ 1544us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.1:37733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 13:07:47.657166 22068 catalog_manager.cc:4584] TS c96fb0fbf59d44bcab77d5030d2cd497 (127.2.86.1:37733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e18005a87c9041e5bdff99ec029b416e: Network error: Client connection negotiation failed: client connection to 127.2.86.1:37733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 13:07:47.657217 22569 ts_tablet_manager.cc:1918] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 13:07:47.657274 22569 log.cc:1196] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Deleting WAL directory at /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790427826505950-2392-0/minicluster-data/ts-2/wal/wals/e18005a87c9041e5bdff99ec029b416e
I20260926 13:07:47.657560 22569 ts_tablet_manager.cc:1939] T e18005a87c9041e5bdff99ec029b416e P 58515146ef2a46be825c8876cba72473: Deleting consensus metadata
I20260926 13:07:47.657975 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22273
I20260926 13:07:47.657964 22068 catalog_manager.cc:4841] TS 58515146ef2a46be825c8876cba72473 (127.2.86.3:39719): tablet e18005a87c9041e5bdff99ec029b416e (table test-table [id=57b878fa79924aa3ba88394518c5648d]) successfully deleted
I20260926 13:07:47.664559 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22404
I20260926 13:07:47.670008 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22051
2026-09-26T13:07:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (621 ms)
[----------] 6 tests from EncryptionPolicyTest (8041 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:47 dist-test-slave-j4lq krb5kdc[22575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:47 dist-test-slave-j4lq krb5kdc[22575](info): set up 2 sockets
Sep 26 13:07:47 dist-test-slave-j4lq krb5kdc[22575](info): commencing operation
krb5kdc: starting...
W20260926 13:07:48.705029 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:48 dist-test-slave-j4lq krb5kdc[22575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:48.817989 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:48.896911 22588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:48.897137 22588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:48.897161 22588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:48.898279 22588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:48.898327 22588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:48.898342 22588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:48.898355 22588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:48.898403 22588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:48.899825 22588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:48.900058 22588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:48.900259 22588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:48.904855 22593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:48.904963 22588 server_base.cc:1034] running on GCE node
W20260926 13:07:48.904855 22596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:48.904909 22594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:48.905328 22588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:48.905629 22588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:48.906800 22588 hybrid_clock.cc:648] HybridClock initialized: now 1790428068906776 us; error 35 us; skew 500 ppm
Sep 26 13:07:48 dist-test-slave-j4lq krb5kdc[22575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428068, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:48.916416 22588 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:48.916957 22588 webserver.cc:460] Webserver started at http://127.0.0.1:36201/ using document root <none> and password file <none>
I20260926 13:07:48.917203 22588 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:48.917272 22588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:48.917415 22588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:48.918452 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "d24e558f81ab4f8186af01853727ad0c"
format_stamp: "Formatted at 2026-09-26 13:07:48 on dist-test-slave-j4lq"
server_key: "25774fc8e73e4834f0e7b364dda90e94"
server_key_iv: "db1f8a29e86b6d654616df69b0342301"
server_key_version: "encryptionkey@0"
I20260926 13:07:48.918812 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "d24e558f81ab4f8186af01853727ad0c"
format_stamp: "Formatted at 2026-09-26 13:07:48 on dist-test-slave-j4lq"
server_key: "25774fc8e73e4834f0e7b364dda90e94"
server_key_iv: "db1f8a29e86b6d654616df69b0342301"
server_key_version: "encryptionkey@0"
I20260926 13:07:48.920188 22588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:48.920989 22603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:48.921201 22588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:07:48.921301 22588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "d24e558f81ab4f8186af01853727ad0c"
format_stamp: "Formatted at 2026-09-26 13:07:48 on dist-test-slave-j4lq"
server_key: "25774fc8e73e4834f0e7b364dda90e94"
server_key_iv: "db1f8a29e86b6d654616df69b0342301"
server_key_version: "encryptionkey@0"
I20260926 13:07:48.921380 22588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:48.940141 22588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:48.942477 22588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:48.942644 22588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:48.946578 22588 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36385
I20260926 13:07:48.946642 22655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36385 every 8 connection(s)
I20260926 13:07:48.946970 22588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:48.947969 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:48.950220 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c: Bootstrap starting.
I20260926 13:07:48.950791 22656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:48.951078 22656 log.cc:824] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:48.951761 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c: No bootstrap required, opened a new log
I20260926 13:07:48.952574 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22588
I20260926 13:07:48.952696 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:48.952939 2392 external_mini_cluster.cc:1428] Setting key 0f5d65e2cd14621edacd994ef78324be
I20260926 13:07:48.953959 22656 raft_consensus.cc:348] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } }
I20260926 13:07:48.954131 22656 raft_consensus.cc:374] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:48.954172 22656 raft_consensus.cc:729] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24e558f81ab4f8186af01853727ad0c, State: Initialized, Role: FOLLOWER
I20260926 13:07:48.954311 22656 consensus_queue.cc:260] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } }
I20260926 13:07:48.954396 22656 raft_consensus.cc:388] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:48.954427 22656 raft_consensus.cc:482] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:48.954478 22656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:48.955461 22656 raft_consensus.cc:504] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } }
I20260926 13:07:48.955550 22656 leader_election.cc:302] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d24e558f81ab4f8186af01853727ad0c; no voters:
I20260926 13:07:48.955757 22656 leader_election.cc:288] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:48.955801 22661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:48.955926 22661 raft_consensus.cc:686] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [term 1 LEADER]: Becoming Leader. State: Replica: d24e558f81ab4f8186af01853727ad0c, State: Running, Role: LEADER
I20260926 13:07:48.956013 22661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } }
I20260926 13:07:48.956346 22661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d24e558f81ab4f8186af01853727ad0c. Latest consensus state: current_term: 1 leader_uuid: "d24e558f81ab4f8186af01853727ad0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } } }
I20260926 13:07:48.956353 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d24e558f81ab4f8186af01853727ad0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24e558f81ab4f8186af01853727ad0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36385 } } }
I20260926 13:07:48.956439 22661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:48.956444 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:48.956020 22656 sys_catalog.cc:564] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:48.956988 22665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:48.957521 22665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:48.959405 22665 catalog_manager.cc:1347] Generated new cluster ID: 3a8a64af2d364061a4f249bdadc0a041
I20260926 13:07:48.959455 22665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:07:48 dist-test-slave-j4lq krb5kdc[22575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:07:48.964669 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:48.953458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46977 (local address 127.0.0.1:36385)
0926 13:07:48.953630 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:48.953645 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:48.953670 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:07:48.953919 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:48.953924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:48.953966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:48.954073 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:48.954883 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:48.955984 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:48.957083 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:48.957692 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:48.958128 (+ 436us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:48.960797 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:48.960801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:48.960802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:48.960813 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:48.962774 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:48.963233 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:48.963236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:48.963236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:48.963359 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:48.963604 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:48.963605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:48.963605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:48.963883 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:48.964027 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:48.964179 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:07:48.964275 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260926 13:07:48.965667 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:48.974305 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:48.967033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57333 (local address 127.0.0.1:36385)
0926 13:07:48.967072 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:48.967074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:48.967249 (+ 175us) server_negotiation.cc:407] Connection header received
0926 13:07:48.967275 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:48.967276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:48.967286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:48.967353 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:48.967942 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:48.968712 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:48.969701 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:48.970049 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:48.970133 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:48.971448 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:48.971450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:48.971450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:48.971457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:48.973149 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:48.973516 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:48.973518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:48.973518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:48.973576 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:48.973788 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:48.973789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:48.973789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:48.973919 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:48.974022 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:48.974148 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 13:07:48.974209 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:07:48.976593 22665 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:48.977320 22665 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:48.983968 22665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d24e558f81ab4f8186af01853727ad0c: Generated new TSK 0
I20260926 13:07:48.984161 22665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:48.987327 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22588
W20260926 13:07:48.992054 22573 connection.cc:535] client connection to 127.0.0.1:36385 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36385): Connection reset by peer (error 104)
2026-09-26T13:07:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:49 dist-test-slave-j4lq krb5kdc[22687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:49 dist-test-slave-j4lq krb5kdc[22687](info): set up 2 sockets
Sep 26 13:07:49 dist-test-slave-j4lq krb5kdc[22687](info): commencing operation
krb5kdc: starting...
W20260926 13:07:50.036942 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:50 dist-test-slave-j4lq krb5kdc[22687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:50.151938 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:50.232182 22700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:50.232401 22700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:50.232443 22700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:50.233651 22700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:50.233717 22700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:50.233741 22700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:50.233768 22700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:50.233794 22700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:50.235248 22700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:50.235512 22700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:50.235739 22700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:50.240334 22708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:50.240367 22706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:50.240338 22705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:50.240605 22700 server_base.cc:1034] running on GCE node
I20260926 13:07:50.240805 22700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:50.241118 22700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:50.242281 22700 hybrid_clock.cc:648] HybridClock initialized: now 1790428070242267 us; error 44 us; skew 500 ppm
Sep 26 13:07:50 dist-test-slave-j4lq krb5kdc[22687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:50.244647 22700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:50.245115 22700 webserver.cc:460] Webserver started at http://127.0.0.1:39087/ using document root <none> and password file <none>
I20260926 13:07:50.245358 22700 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:50.245424 22700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:50.245565 22700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:50.246636 22700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6"
format_stamp: "Formatted at 2026-09-26 13:07:50 on dist-test-slave-j4lq"
server_key: "aa03eeb78e6ffb95b8be60d8cf38400d"
server_key_iv: "b1309a2f50873c9f673f3f9117f7d51f"
server_key_version: "encryptionkey@0"
I20260926 13:07:50.246984 22700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6"
format_stamp: "Formatted at 2026-09-26 13:07:50 on dist-test-slave-j4lq"
server_key: "aa03eeb78e6ffb95b8be60d8cf38400d"
server_key_iv: "b1309a2f50873c9f673f3f9117f7d51f"
server_key_version: "encryptionkey@0"
I20260926 13:07:50.248246 22700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:50.248970 22715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:50.249138 22700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:50.249233 22700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6"
format_stamp: "Formatted at 2026-09-26 13:07:50 on dist-test-slave-j4lq"
server_key: "aa03eeb78e6ffb95b8be60d8cf38400d"
server_key_iv: "b1309a2f50873c9f673f3f9117f7d51f"
server_key_version: "encryptionkey@0"
I20260926 13:07:50.249312 22700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:50.270397 22700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:50.272369 22700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:50.272531 22700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:50.276473 22700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41723
I20260926 13:07:50.276525 22767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41723 every 8 connection(s)
I20260926 13:07:50.276914 22700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:50.277945 22768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:50.280121 22768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6: Bootstrap starting.
I20260926 13:07:50.280692 22768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:50.281026 22768 log.cc:824] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:50.281688 22768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6: No bootstrap required, opened a new log
I20260926 13:07:50.283238 22768 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } }
I20260926 13:07:50.283370 22768 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:50.283418 22768 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc39eac8a1a3407ab4fbfac37b63e2f6, State: Initialized, Role: FOLLOWER
I20260926 13:07:50.283572 22768 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } }
I20260926 13:07:50.283653 22768 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:50.283704 22768 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:50.283756 22768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:50.284289 22768 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } }
I20260926 13:07:50.284399 22768 leader_election.cc:302] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc39eac8a1a3407ab4fbfac37b63e2f6; no voters:
I20260926 13:07:50.284581 22768 leader_election.cc:288] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:50.284610 22771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:50.284782 22771 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [term 1 LEADER]: Becoming Leader. State: Replica: fc39eac8a1a3407ab4fbfac37b63e2f6, State: Running, Role: LEADER
I20260926 13:07:50.284818 22768 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:50.284909 22771 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } }
I20260926 13:07:50.285301 22773 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc39eac8a1a3407ab4fbfac37b63e2f6. Latest consensus state: current_term: 1 leader_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } } }
I20260926 13:07:50.285431 22772 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc39eac8a1a3407ab4fbfac37b63e2f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41723 } } }
I20260926 13:07:50.285442 22773 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:50.285534 22772 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:50.285861 22780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:50.286371 22780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:50.286491 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22700
I20260926 13:07:50.286618 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:50.286866 2392 external_mini_cluster.cc:1428] Setting key 8029c49da445d1bf92944af2e5126a27
I20260926 13:07:50.288590 22780 catalog_manager.cc:1347] Generated new cluster ID: bbe557206421478abfebe964da7d890b
I20260926 13:07:50.288647 22780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:50.292233 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:50.287310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54355 (local address 127.0.0.1:41723)
0926 13:07:50.287805 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:50.287814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:50.287845 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:07:50.287954 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:50.287960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:50.288012 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:50.288165 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:50.288396 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:50.289494 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:50.290304 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:50.290748 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:50.290882 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:50.291606 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:50.291609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:50.291610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:50.291784 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:50.291813 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:50.291824 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:50.291915 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:50.291993 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 13:07:50.292022 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":65,"threads_started":1}
I20260926 13:07:50.293423 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:50.304064 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:50.301180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51855 (local address 127.0.0.1:41723)
0926 13:07:50.301225 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:50.301227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:50.301362 (+ 135us) server_negotiation.cc:407] Connection header received
0926 13:07:50.301425 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:50.301426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:50.301437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:50.301492 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:50.301656 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:50.302136 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:50.302843 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:50.303063 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:50.303152 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:50.303826 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:50.303827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:50.303828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:50.303834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:50.303847 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:50.303851 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:50.303905 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:50.303972 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 13:07:50.303983 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 13:07:50.309293 22780 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:50.309805 22780 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:50.313360 22780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc39eac8a1a3407ab4fbfac37b63e2f6: Generated new TSK 0
I20260926 13:07:50.313540 22780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:50.320051 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22700
W20260926 13:07:50.324164 22685 connection.cc:535] client connection to 127.0.0.1:41723 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41723): Connection reset by peer (error 104)
2026-09-26T13:07:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:50 dist-test-slave-j4lq krb5kdc[22799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:50 dist-test-slave-j4lq krb5kdc[22799](info): set up 2 sockets
Sep 26 13:07:50 dist-test-slave-j4lq krb5kdc[22799](info): commencing operation
krb5kdc: starting...
W20260926 13:07:51.349073 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:51 dist-test-slave-j4lq krb5kdc[22799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:51.461547 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:51.542902 22812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:51.543110 22812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:51.543133 22812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:51.544250 22812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:51.544287 22812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:51.544301 22812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:51.544314 22812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:51.544327 22812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:51.545799 22812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:51.545998 22812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:51.546169 22812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:51.550927 22817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:51.550900 22818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:51.551100 22812 server_base.cc:1034] running on GCE node
W20260926 13:07:51.550915 22820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:51.551379 22812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:51.551667 22812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:51.552806 22812 hybrid_clock.cc:648] HybridClock initialized: now 1790428071552800 us; error 31 us; skew 500 ppm
Sep 26 13:07:51 dist-test-slave-j4lq krb5kdc[22799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428071, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:51.555228 22812 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:51.555686 22812 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
I20260926 13:07:51.555917 22812 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:51.555981 22812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:51.556129 22812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:51.557204 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "284d0b4ea9444c24910356a5ff214588"
format_stamp: "Formatted at 2026-09-26 13:07:51 on dist-test-slave-j4lq"
server_key: "747193e0c824c6273feb7d5e91116aca"
server_key_iv: "72e6d4be4e7de9948c4cf0476b985539"
server_key_version: "encryptionkey@0"
I20260926 13:07:51.557543 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "284d0b4ea9444c24910356a5ff214588"
format_stamp: "Formatted at 2026-09-26 13:07:51 on dist-test-slave-j4lq"
server_key: "747193e0c824c6273feb7d5e91116aca"
server_key_iv: "72e6d4be4e7de9948c4cf0476b985539"
server_key_version: "encryptionkey@0"
I20260926 13:07:51.558800 22812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:51.559486 22827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:51.559659 22812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:51.559759 22812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "284d0b4ea9444c24910356a5ff214588"
format_stamp: "Formatted at 2026-09-26 13:07:51 on dist-test-slave-j4lq"
server_key: "747193e0c824c6273feb7d5e91116aca"
server_key_iv: "72e6d4be4e7de9948c4cf0476b985539"
server_key_version: "encryptionkey@0"
I20260926 13:07:51.559830 22812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:51.563056 22812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:51.565266 22812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:51.565418 22812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:51.569408 22812 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34627
I20260926 13:07:51.569437 22879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34627 every 8 connection(s)
I20260926 13:07:51.569758 22812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:51.570638 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:51.572693 22880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588: Bootstrap starting.
I20260926 13:07:51.573271 22880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:51.573498 22880 log.cc:824] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:51.574076 22880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588: No bootstrap required, opened a new log
I20260926 13:07:51.575551 22880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } }
I20260926 13:07:51.575677 22880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:51.575700 22880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284d0b4ea9444c24910356a5ff214588, State: Initialized, Role: FOLLOWER
I20260926 13:07:51.575834 22880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } }
I20260926 13:07:51.575910 22880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:51.575932 22880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:51.575984 22880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:51.576149 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22812
I20260926 13:07:51.576268 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:51.576479 2392 external_mini_cluster.cc:1428] Setting key 5e5bb9cae20eec0d15c15774bb3b40e0
I20260926 13:07:51.576697 22880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } }
I20260926 13:07:51.576808 22880 leader_election.cc:302] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284d0b4ea9444c24910356a5ff214588; no voters:
I20260926 13:07:51.577011 22880 leader_election.cc:288] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:51.577040 22884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:51.577268 22884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [term 1 LEADER]: Becoming Leader. State: Replica: 284d0b4ea9444c24910356a5ff214588, State: Running, Role: LEADER
I20260926 13:07:51.577355 22884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } }
I20260926 13:07:51.577386 22880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:51.577646 22884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284d0b4ea9444c24910356a5ff214588. Latest consensus state: current_term: 1 leader_uuid: "284d0b4ea9444c24910356a5ff214588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } } }
I20260926 13:07:51.577657 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284d0b4ea9444c24910356a5ff214588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d0b4ea9444c24910356a5ff214588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34627 } } }
I20260926 13:07:51.577824 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:51.577726 22884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:51.578753 22890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:51.579232 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:51.576947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59819 (local address 127.0.0.1:34627)
0926 13:07:51.577142 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:51.577151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:51.577246 (+ 95us) server_negotiation.cc:407] Connection header received
0926 13:07:51.577361 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:51.577368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:51.577397 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:51.577561 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:51.578665 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:51.578668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:51.578668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:51.578673 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:51.578718 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:51.578732 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:51.578958 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:51.578990 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 13:07:51.579026 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260926 13:07:51.579232 22890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:51.580088 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:51.580976 22890 catalog_manager.cc:1347] Generated new cluster ID: 1a09263b57f3439dbfdd70d0e2ae253a
I20260926 13:07:51.581027 22890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:51.585894 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:51.581695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41019 (local address 127.0.0.1:34627)
0926 13:07:51.584961 (+ 3266us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:51.584963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:51.584971 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:07:51.584991 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:51.584992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:51.584996 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:51.585078 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:51.585653 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:51.585654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:51.585655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:51.585659 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:51.585672 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:51.585675 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:51.585732 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:51.585781 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 13:07:51.585795 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3246}
I20260926 13:07:51.606504 22890 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:51.606711 22890 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:51.611387 22890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284d0b4ea9444c24910356a5ff214588: Generated new TSK 0
I20260926 13:07:51.611594 22890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:51.619227 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22812
2026-09-26T13:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:51 dist-test-slave-j4lq krb5kdc[22911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:51 dist-test-slave-j4lq krb5kdc[22911](info): set up 2 sockets
Sep 26 13:07:51 dist-test-slave-j4lq krb5kdc[22911](info): commencing operation
krb5kdc: starting...
W20260926 13:07:52.648905 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[22911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:52.770183 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32973
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:52.850721 22924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:52.850970 22924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:52.851011 22924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:52.852166 22924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:52.852239 22924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:52.852263 22924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:52.852291 22924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:52.852322 22924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:52.853809 22924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:52.854094 22924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:52.854332 22924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:52.859143 22930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:52.859197 22929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:52.859143 22932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:52.859418 22924 server_base.cc:1034] running on GCE node
I20260926 13:07:52.859696 22924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:52.860000 22924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:52.861166 22924 hybrid_clock.cc:648] HybridClock initialized: now 1790428072861146 us; error 37 us; skew 500 ppm
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[22911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428072, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:52.863575 22924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:52.864044 22924 webserver.cc:460] Webserver started at http://127.0.0.1:40869/ using document root <none> and password file <none>
I20260926 13:07:52.864275 22924 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:52.864341 22924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:52.864492 22924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:52.865603 22924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "4207292fd0c5452eb5c9d50b7d279c9d"
format_stamp: "Formatted at 2026-09-26 13:07:52 on dist-test-slave-j4lq"
server_key: "a6a51e2cdcbab2aa54ca30fc5244f7ab"
server_key_iv: "338e3c58033b551ce316ddc273c13d44"
server_key_version: "encryptionkey@0"
I20260926 13:07:52.865994 22924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "4207292fd0c5452eb5c9d50b7d279c9d"
format_stamp: "Formatted at 2026-09-26 13:07:52 on dist-test-slave-j4lq"
server_key: "a6a51e2cdcbab2aa54ca30fc5244f7ab"
server_key_iv: "338e3c58033b551ce316ddc273c13d44"
server_key_version: "encryptionkey@0"
I20260926 13:07:52.867306 22924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:52.868121 22939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:52.868314 22924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:52.868417 22924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "4207292fd0c5452eb5c9d50b7d279c9d"
format_stamp: "Formatted at 2026-09-26 13:07:52 on dist-test-slave-j4lq"
server_key: "a6a51e2cdcbab2aa54ca30fc5244f7ab"
server_key_iv: "338e3c58033b551ce316ddc273c13d44"
server_key_version: "encryptionkey@0"
I20260926 13:07:52.868495 22924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:52.879978 22924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:52.881884 22924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:52.882061 22924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:52.886098 22924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32973
I20260926 13:07:52.886132 22991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32973 every 8 connection(s)
I20260926 13:07:52.886513 22924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:52.887466 22992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:52.889782 22992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d: Bootstrap starting.
I20260926 13:07:52.890328 22992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:52.890555 22992 log.cc:824] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:52.891240 22992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d: No bootstrap required, opened a new log
I20260926 13:07:52.892764 22992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } }
I20260926 13:07:52.892968 22992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:52.892997 22992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4207292fd0c5452eb5c9d50b7d279c9d, State: Initialized, Role: FOLLOWER
I20260926 13:07:52.893107 22992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } }
I20260926 13:07:52.893168 22992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:52.893188 22992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:52.893216 22992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:52.893682 22992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } }
I20260926 13:07:52.893769 22992 leader_election.cc:302] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4207292fd0c5452eb5c9d50b7d279c9d; no voters:
I20260926 13:07:52.893882 22992 leader_election.cc:288] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:52.893944 22995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:52.894119 22995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [term 1 LEADER]: Becoming Leader. State: Replica: 4207292fd0c5452eb5c9d50b7d279c9d, State: Running, Role: LEADER
I20260926 13:07:52.894181 22992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:52.894249 22995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } }
I20260926 13:07:52.894632 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4207292fd0c5452eb5c9d50b7d279c9d. Latest consensus state: current_term: 1 leader_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } } }
I20260926 13:07:52.894629 22996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4207292fd0c5452eb5c9d50b7d279c9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32973 } } }
I20260926 13:07:52.894748 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:52.894812 22996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:52.895032 22999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:52.895180 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 22924
I20260926 13:07:52.895305 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:52.895509 2392 external_mini_cluster.cc:1428] Setting key 8c8f3406f69098807ee01ad6786edd81
I20260926 13:07:52.895596 22999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:52.897320 22999 catalog_manager.cc:1347] Generated new cluster ID: cd1f217a9ca64809b3c2e67b04710d8f
I20260926 13:07:52.897398 22999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[22911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:07:52.906903 22999 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:52.907599 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:52.896107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53249 (local address 127.0.0.1:32973)
0926 13:07:52.896498 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:52.896507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:52.896535 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:07:52.896645 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:52.896663 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:52.896727 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:52.896911 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:52.897679 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:52.899039 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:52.900163 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:52.900747 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:52.900840 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:52.903751 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:52.903756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:52.903756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:52.903769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:52.905807 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:52.906232 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:52.906235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:52.906235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:52.906345 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:52.906595 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:52.906596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:52.906596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:52.906902 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:52.907062 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:52.907166 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 13:07:52.907420 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":64,"threads_started":1}
I20260926 13:07:52.907687 22999 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:52.908499 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:52.912333 22999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4207292fd0c5452eb5c9d50b7d279c9d: Generated new TSK 0
I20260926 13:07:52.912500 22999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:52.919304 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:52.909888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38919 (local address 127.0.0.1:32973)
0926 13:07:52.909928 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:52.909929 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:52.912649 (+ 2720us) server_negotiation.cc:407] Connection header received
0926 13:07:52.912737 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:52.912738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:52.912763 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:52.912822 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:52.913392 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:52.914188 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:52.914893 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:52.915256 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:52.915327 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:52.916497 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:52.916499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:52.916499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:52.916505 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:52.918114 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:52.918461 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:52.918462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:52.918462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:52.918520 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:52.918761 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:52.918762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:52.918762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:52.918894 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:52.919000 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:52.919132 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 13:07:52.919209 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:07:52.921332 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 22924
2026-09-26T13:07:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[23023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[23023](info): set up 2 sockets
Sep 26 13:07:52 dist-test-slave-j4lq krb5kdc[23023](info): commencing operation
krb5kdc: starting...
W20260926 13:07:53.949029 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:54 dist-test-slave-j4lq krb5kdc[23023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:54.063807 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:54.143169 23036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:54.143419 23036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:54.143466 23036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:54.144636 23036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:54.144707 23036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:54.144733 23036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:54.144766 23036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:54.144789 23036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:54.146307 23036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:54.146595 23036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:54.146823 23036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:54.151508 23042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:54.151562 23041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:54.151562 23044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:54.151800 23036 server_base.cc:1034] running on GCE node
I20260926 13:07:54.152073 23036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:54.152355 23036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:54.153517 23036 hybrid_clock.cc:648] HybridClock initialized: now 1790428074153512 us; error 35 us; skew 500 ppm
Sep 26 13:07:54 dist-test-slave-j4lq krb5kdc[23023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428074, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:54.159507 23036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:54.159973 23036 webserver.cc:460] Webserver started at http://127.0.0.1:34347/ using document root <none> and password file <none>
I20260926 13:07:54.160182 23036 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:54.160223 23036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:54.160310 23036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:54.161381 23036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "ba3610bc87d34d9bba896458e3b1198e"
format_stamp: "Formatted at 2026-09-26 13:07:54 on dist-test-slave-j4lq"
server_key: "276e5bd4e668a78708a870ea338a0b32"
server_key_iv: "5070b00b86bcc3fe897c2dd913940e25"
server_key_version: "encryptionkey@0"
I20260926 13:07:54.161679 23036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "ba3610bc87d34d9bba896458e3b1198e"
format_stamp: "Formatted at 2026-09-26 13:07:54 on dist-test-slave-j4lq"
server_key: "276e5bd4e668a78708a870ea338a0b32"
server_key_iv: "5070b00b86bcc3fe897c2dd913940e25"
server_key_version: "encryptionkey@0"
I20260926 13:07:54.162922 23036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:07:54.163700 23051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:54.164032 23036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:54.164119 23036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "ba3610bc87d34d9bba896458e3b1198e"
format_stamp: "Formatted at 2026-09-26 13:07:54 on dist-test-slave-j4lq"
server_key: "276e5bd4e668a78708a870ea338a0b32"
server_key_iv: "5070b00b86bcc3fe897c2dd913940e25"
server_key_version: "encryptionkey@0"
I20260926 13:07:54.164170 23036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:54.192101 23036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:54.194514 23036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:54.194629 23036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:54.198812 23036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37645
I20260926 13:07:54.198879 23103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37645 every 8 connection(s)
I20260926 13:07:54.199208 23036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:54.200184 23104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:54.202499 23104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e: Bootstrap starting.
I20260926 13:07:54.203025 23104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:54.203265 23104 log.cc:824] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:54.203917 23104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e: No bootstrap required, opened a new log
I20260926 13:07:54.205552 23104 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } }
I20260926 13:07:54.205689 23104 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:54.205711 23104 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3610bc87d34d9bba896458e3b1198e, State: Initialized, Role: FOLLOWER
I20260926 13:07:54.205842 23104 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } }
I20260926 13:07:54.205924 23104 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:54.205945 23104 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:54.205973 23104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:54.206442 23104 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } }
I20260926 13:07:54.206523 23104 leader_election.cc:302] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba3610bc87d34d9bba896458e3b1198e; no voters:
I20260926 13:07:54.206645 23104 leader_election.cc:288] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:54.206722 23107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:54.206900 23107 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [term 1 LEADER]: Becoming Leader. State: Replica: ba3610bc87d34d9bba896458e3b1198e, State: Running, Role: LEADER
I20260926 13:07:54.206964 23104 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:54.207020 23107 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } }
I20260926 13:07:54.207458 23109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba3610bc87d34d9bba896458e3b1198e. Latest consensus state: current_term: 1 leader_uuid: "ba3610bc87d34d9bba896458e3b1198e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } } }
I20260926 13:07:54.207562 23109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:54.207691 23108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba3610bc87d34d9bba896458e3b1198e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3610bc87d34d9bba896458e3b1198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37645 } } }
I20260926 13:07:54.207783 23108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:54.207974 23116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:54.208523 23116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:54.208712 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23036
I20260926 13:07:54.208827 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:54.209117 2392 external_mini_cluster.cc:1428] Setting key 0d4471fecc428dad22825ac019a02118
I20260926 13:07:54.210543 23116 catalog_manager.cc:1347] Generated new cluster ID: ae8076789e004bfca4f41c40f0c43eee
I20260926 13:07:54.210594 23116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:54.217808 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:54.209754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59539 (local address 127.0.0.1:37645)
0926 13:07:54.209937 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:54.209948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:54.210096 (+ 148us) server_negotiation.cc:407] Connection header received
0926 13:07:54.210248 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:54.210255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:54.210311 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:54.210447 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:54.213000 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:54.214388 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:54.215522 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:54.216123 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:54.216222 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:54.217014 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:54.217018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:54.217019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:54.217230 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:54.217266 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:54.217281 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:54.217341 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:54.217434 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 13:07:54.217585 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260926 13:07:54.218636 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:54.223472 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:54.220186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49737 (local address 127.0.0.1:37645)
0926 13:07:54.220215 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:54.220216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:54.220299 (+ 83us) server_negotiation.cc:407] Connection header received
0926 13:07:54.220379 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:54.220379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:54.220389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:54.220414 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:54.220569 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:54.221291 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:54.222239 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:54.222538 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:54.222572 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:54.223249 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:54.223250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:54.223251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:54.223254 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:54.223265 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:54.223268 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:54.223292 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:54.223383 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 13:07:54.223405 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 13:07:54.229079 23116 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:54.229818 23116 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:54.238843 23116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba3610bc87d34d9bba896458e3b1198e: Generated new TSK 0
I20260926 13:07:54.239035 23116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:54.243464 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23036
2026-09-26T13:07:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:54 dist-test-slave-j4lq krb5kdc[23135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:54 dist-test-slave-j4lq krb5kdc[23135](info): set up 2 sockets
Sep 26 13:07:54 dist-test-slave-j4lq krb5kdc[23135](info): commencing operation
krb5kdc: starting...
W20260926 13:07:55.280989 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:55 dist-test-slave-j4lq krb5kdc[23135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:55.393229 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:55.473482 23148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:55.473765 23148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:55.473819 23148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:55.474997 23148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:55.475069 23148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:55.475095 23148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:55.475126 23148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:55.475147 23148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:55.476688 23148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:55.477161 23148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:55.477363 23148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:55.482141 23156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:55.482272 23153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:55.482221 23154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:55.482517 23148 server_base.cc:1034] running on GCE node
I20260926 13:07:55.482780 23148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:55.483114 23148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:55.484273 23148 hybrid_clock.cc:648] HybridClock initialized: now 1790428075484284 us; error 66 us; skew 500 ppm
Sep 26 13:07:55 dist-test-slave-j4lq krb5kdc[23135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:55.486775 23148 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:55.487257 23148 webserver.cc:460] Webserver started at http://127.0.0.1:41407/ using document root <none> and password file <none>
I20260926 13:07:55.487483 23148 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:55.487546 23148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:55.487700 23148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:55.488765 23148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "a8d6da16ef154a84927ce746b9853d08"
format_stamp: "Formatted at 2026-09-26 13:07:55 on dist-test-slave-j4lq"
server_key: "4464a9549b305d07c937e8268a717f3f"
server_key_iv: "a73a4cba6ed769755e0287dffafa0f64"
server_key_version: "encryptionkey@0"
I20260926 13:07:55.489197 23148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "a8d6da16ef154a84927ce746b9853d08"
format_stamp: "Formatted at 2026-09-26 13:07:55 on dist-test-slave-j4lq"
server_key: "4464a9549b305d07c937e8268a717f3f"
server_key_iv: "a73a4cba6ed769755e0287dffafa0f64"
server_key_version: "encryptionkey@0"
I20260926 13:07:55.490530 23148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:07:55.491257 23163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:55.491429 23148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:55.491524 23148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "a8d6da16ef154a84927ce746b9853d08"
format_stamp: "Formatted at 2026-09-26 13:07:55 on dist-test-slave-j4lq"
server_key: "4464a9549b305d07c937e8268a717f3f"
server_key_iv: "a73a4cba6ed769755e0287dffafa0f64"
server_key_version: "encryptionkey@0"
I20260926 13:07:55.491605 23148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:55.494968 23148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:55.496795 23148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:55.496980 23148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:55.500965 23148 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35561
I20260926 13:07:55.501008 23215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35561 every 8 connection(s)
I20260926 13:07:55.501346 23148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:55.502243 23216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:55.504374 23216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08: Bootstrap starting.
I20260926 13:07:55.504931 23216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:55.505177 23216 log.cc:824] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:55.505784 23216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08: No bootstrap required, opened a new log
I20260926 13:07:55.507269 23216 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260926 13:07:55.507442 23216 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:55.507490 23216 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d6da16ef154a84927ce746b9853d08, State: Initialized, Role: FOLLOWER
I20260926 13:07:55.507547 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23148
I20260926 13:07:55.507664 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:55.507668 23216 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260926 13:07:55.507755 23216 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:55.507795 23216 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:55.507845 23216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:55.507884 2392 external_mini_cluster.cc:1428] Setting key 6e4e837eb11a772de31dc20ca05b5515
I20260926 13:07:55.508540 23216 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260926 13:07:55.508647 23216 leader_election.cc:302] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8d6da16ef154a84927ce746b9853d08; no voters:
I20260926 13:07:55.508821 23216 leader_election.cc:288] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:55.508855 23221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:55.509042 23221 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [term 1 LEADER]: Becoming Leader. State: Replica: a8d6da16ef154a84927ce746b9853d08, State: Running, Role: LEADER
I20260926 13:07:55.509091 23216 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:55.509220 23221 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } }
I20260926 13:07:55.509688 23222 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8d6da16ef154a84927ce746b9853d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } } }
I20260926 13:07:55.509671 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d6da16ef154a84927ce746b9853d08. Latest consensus state: current_term: 1 leader_uuid: "a8d6da16ef154a84927ce746b9853d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d6da16ef154a84927ce746b9853d08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35561 } } }
I20260926 13:07:55.509804 23222 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:55.509809 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:55.510011 23230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:55.510556 23230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:55.511224 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:55.508592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37943 (local address 127.0.0.1:35561)
0926 13:07:55.508752 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:55.508763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:55.508794 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:07:55.508932 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:55.508939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:55.508970 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:55.509373 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:55.510614 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:55.510617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:55.510618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:55.510623 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:55.510672 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:55.510688 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:55.510764 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:55.510984 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 13:07:55.511028 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260926 13:07:55.512013 23230 catalog_manager.cc:1347] Generated new cluster ID: 8b62c64df6ae43198188d6a85f1ae47e
I20260926 13:07:55.512068 23230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:55.512259 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:55.514748 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:55.513752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57535 (local address 127.0.0.1:35561)
0926 13:07:55.513804 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:55.513806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:55.513869 (+ 63us) server_negotiation.cc:407] Connection header received
0926 13:07:55.513943 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:55.513943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:55.513948 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:55.514007 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:55.514530 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:55.514531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:55.514531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:55.514535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:55.514554 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:55.514558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:55.514614 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:55.514680 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 13:07:55.514692 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:07:55.516618 23230 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:55.516757 23230 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:55.527293 23230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8d6da16ef154a84927ce746b9853d08: Generated new TSK 0
I20260926 13:07:55.527493 23230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:55.530468 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23148
2026-09-26T13:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:55 dist-test-slave-j4lq krb5kdc[23247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:55 dist-test-slave-j4lq krb5kdc[23247](info): set up 2 sockets
Sep 26 13:07:55 dist-test-slave-j4lq krb5kdc[23247](info): commencing operation
krb5kdc: starting...
W20260926 13:07:56.568938 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:56.681205 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:56.760371 23260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:56.760612 23260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:56.760649 23260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:56.761794 23260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:56.761860 23260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:56.761883 23260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:56.761905 23260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:56.761931 23260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:56.763357 23260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:56.763625 23260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:56.763841 23260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:56.768668 23266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:56.768827 23260 server_base.cc:1034] running on GCE node
W20260926 13:07:56.768695 23268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:56.768695 23265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:56.769176 23260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:56.769461 23260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:56.770617 23260 hybrid_clock.cc:648] HybridClock initialized: now 1790428076770613 us; error 39 us; skew 500 ppm
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:56.773418 23260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:56.773898 23260 webserver.cc:460] Webserver started at http://127.0.0.1:42455/ using document root <none> and password file <none>
I20260926 13:07:56.774144 23260 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:56.774214 23260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:56.774369 23260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:56.775488 23260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "81b89620954445a187cd00489ec02827"
format_stamp: "Formatted at 2026-09-26 13:07:56 on dist-test-slave-j4lq"
server_key: "bc40b684d8b3cd5bee757c4f5368adb8"
server_key_iv: "4e1debd2999909259216cc721ec5acda"
server_key_version: "encryptionkey@0"
I20260926 13:07:56.775866 23260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "81b89620954445a187cd00489ec02827"
format_stamp: "Formatted at 2026-09-26 13:07:56 on dist-test-slave-j4lq"
server_key: "bc40b684d8b3cd5bee757c4f5368adb8"
server_key_iv: "4e1debd2999909259216cc721ec5acda"
server_key_version: "encryptionkey@0"
I20260926 13:07:56.777309 23260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:07:56.778093 23275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:56.778254 23260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:07:56.778359 23260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "81b89620954445a187cd00489ec02827"
format_stamp: "Formatted at 2026-09-26 13:07:56 on dist-test-slave-j4lq"
server_key: "bc40b684d8b3cd5bee757c4f5368adb8"
server_key_iv: "4e1debd2999909259216cc721ec5acda"
server_key_version: "encryptionkey@0"
I20260926 13:07:56.778434 23260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:56.798902 23260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:56.801445 23260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:56.801623 23260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:56.806103 23260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38897
I20260926 13:07:56.806139 23327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38897 every 8 connection(s)
I20260926 13:07:56.806528 23260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:56.807479 23328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:56.809767 23328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827: Bootstrap starting.
I20260926 13:07:56.810334 23328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:56.810570 23328 log.cc:824] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:56.811229 23328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827: No bootstrap required, opened a new log
I20260926 13:07:56.812784 23328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260926 13:07:56.812984 23328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:56.813014 23328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b89620954445a187cd00489ec02827, State: Initialized, Role: FOLLOWER
I20260926 13:07:56.813122 23328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260926 13:07:56.813175 23328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:56.813195 23328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:56.813221 23328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:56.813697 23328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260926 13:07:56.813781 23328 leader_election.cc:302] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81b89620954445a187cd00489ec02827; no voters:
I20260926 13:07:56.813894 23328 leader_election.cc:288] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:56.814024 23331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:56.814195 23331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [term 1 LEADER]: Becoming Leader. State: Replica: 81b89620954445a187cd00489ec02827, State: Running, Role: LEADER
I20260926 13:07:56.814262 23328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:56.814327 23331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260926 13:07:56.814751 23333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81b89620954445a187cd00489ec02827. Latest consensus state: current_term: 1 leader_uuid: "81b89620954445a187cd00489ec02827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260926 13:07:56.814848 23333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:56.814971 23332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81b89620954445a187cd00489ec02827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b89620954445a187cd00489ec02827" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260926 13:07:56.815107 23332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:56.815280 23340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:56.815639 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23260
I20260926 13:07:56.815802 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:56.815824 23340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:56.816043 2392 external_mini_cluster.cc:1428] Setting key 966a9caef299e771c45f566579428792
I20260926 13:07:56.817749 23340 catalog_manager.cc:1347] Generated new cluster ID: 9578248b3a264f7e9857e400386f1df7
I20260926 13:07:56.817798 23340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:07:56.831656 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:56.816550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:47371 (local address 127.0.0.1:38897)
0926 13:07:56.816733 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:56.816743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:56.816779 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:07:56.816945 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:56.816952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:56.817009 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:56.817148 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:56.821312 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:56.823105 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:56.824163 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:56.824895 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:56.825027 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:56.828016 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:56.828021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:56.828022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:56.828034 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:56.829968 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:56.830411 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:56.830414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:56.830414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:56.830515 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:56.830741 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:56.830742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:56.830742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:56.831025 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:56.831160 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:56.831335 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 13:07:56.831432 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260926 13:07:56.832708 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:56.841230 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:56.834193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:53439 (local address 127.0.0.1:38897)
0926 13:07:56.834228 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:56.834230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:56.834300 (+ 70us) server_negotiation.cc:407] Connection header received
0926 13:07:56.834356 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:56.834357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:56.834367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:56.834433 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:56.834998 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:56.835758 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:56.836741 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:56.837093 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:56.837163 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:56.838433 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:56.838434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:56.838434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:07:56.838441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:56.840039 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:56.840421 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:56.840422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:56.840422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:56.840482 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:07:56.840700 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:07:56.840701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:07:56.840701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:07:56.840828 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:56.840973 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:56.841074 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 13:07:56.841138 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:07:56.842737 23340 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:56.843215 23340 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:56.847191 23340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81b89620954445a187cd00489ec02827: Generated new TSK 0
I20260926 13:07:56.847378 23340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:56.853618 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23260
W20260926 13:07:56.858457 23245 connection.cc:535] client connection to 127.0.0.1:38897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38897): Connection reset by peer (error 104)
2026-09-26T13:07:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23359](info): set up 2 sockets
Sep 26 13:07:56 dist-test-slave-j4lq krb5kdc[23359](info): commencing operation
krb5kdc: starting...
W20260926 13:07:57.928975 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:58 dist-test-slave-j4lq krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:58.046844 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:58.124567 23372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:58.124807 23372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:58.124846 23372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:58.125993 23372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:58.126060 23372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:58.126085 23372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:58.126116 23372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:58.126143 23372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:58.127565 23372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:58.127832 23372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:58.128047 23372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:58.132706 23377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:58.132704 23380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:58.132788 23378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:58.132840 23372 server_base.cc:1034] running on GCE node
I20260926 13:07:58.133203 23372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:58.133510 23372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:58.134684 23372 hybrid_clock.cc:648] HybridClock initialized: now 1790428078134666 us; error 28 us; skew 500 ppm
Sep 26 13:07:58 dist-test-slave-j4lq krb5kdc[23359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428078, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:58.137166 23372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:58.137573 23372 webserver.cc:460] Webserver started at http://127.0.0.1:46385/ using document root <none> and password file <none>
I20260926 13:07:58.137787 23372 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:58.137849 23372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:58.137980 23372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:58.138983 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "09d67fe2c4f74f379c684c9424739d81"
format_stamp: "Formatted at 2026-09-26 13:07:58 on dist-test-slave-j4lq"
server_key: "098af83bc4fece5f277c038ef2b2e7c4"
server_key_iv: "e2e0dd7e0ee59b54fa8d29ac7e79ab53"
server_key_version: "encryptionkey@0"
I20260926 13:07:58.139339 23372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "09d67fe2c4f74f379c684c9424739d81"
format_stamp: "Formatted at 2026-09-26 13:07:58 on dist-test-slave-j4lq"
server_key: "098af83bc4fece5f277c038ef2b2e7c4"
server_key_iv: "e2e0dd7e0ee59b54fa8d29ac7e79ab53"
server_key_version: "encryptionkey@0"
I20260926 13:07:58.140594 23372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:07:58.141348 23387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:58.141497 23372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:58.141589 23372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "09d67fe2c4f74f379c684c9424739d81"
format_stamp: "Formatted at 2026-09-26 13:07:58 on dist-test-slave-j4lq"
server_key: "098af83bc4fece5f277c038ef2b2e7c4"
server_key_iv: "e2e0dd7e0ee59b54fa8d29ac7e79ab53"
server_key_version: "encryptionkey@0"
I20260926 13:07:58.141669 23372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:58.161140 23372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:58.162993 23372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:58.163146 23372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:58.166997 23372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45193
I20260926 13:07:58.167037 23439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45193 every 8 connection(s)
I20260926 13:07:58.167387 23372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:58.168324 23440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:58.170562 23440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81: Bootstrap starting.
I20260926 13:07:58.170872 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23372
I20260926 13:07:58.170984 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:58.171171 2392 external_mini_cluster.cc:1428] Setting key 23a0d211eed4e4750d5629a4d898cdee
I20260926 13:07:58.171227 23440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:58.171566 23440 log.cc:824] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:58.172354 23440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81: No bootstrap required, opened a new log
I20260926 13:07:58.174809 23440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } }
I20260926 13:07:58.174940 23440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:58.174961 23440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d67fe2c4f74f379c684c9424739d81, State: Initialized, Role: FOLLOWER
I20260926 13:07:58.175101 23440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } }
I20260926 13:07:58.175172 23440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:58.175212 23440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:58.175261 23440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:58.175920 23440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } }
I20260926 13:07:58.176033 23440 leader_election.cc:302] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09d67fe2c4f74f379c684c9424739d81; no voters:
I20260926 13:07:58.176189 23440 leader_election.cc:288] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:58.176229 23445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:58.176415 23445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [term 1 LEADER]: Becoming Leader. State: Replica: 09d67fe2c4f74f379c684c9424739d81, State: Running, Role: LEADER
I20260926 13:07:58.176436 23440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:58.176561 23445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } }
I20260926 13:07:58.176994 23447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d67fe2c4f74f379c684c9424739d81. Latest consensus state: current_term: 1 leader_uuid: "09d67fe2c4f74f379c684c9424739d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } } }
I20260926 13:07:58.177078 23447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:58.177388 23454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:58.177424 23446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09d67fe2c4f74f379c684c9424739d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d67fe2c4f74f379c684c9424739d81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45193 } } }
I20260926 13:07:58.177556 23446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:58.177744 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:58.171690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:41969 (local address 127.0.0.1:45193)
0926 13:07:58.171864 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:58.171872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:58.171896 (+ 24us) server_negotiation.cc:407] Connection header received
0926 13:07:58.172482 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:58.172493 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:58.172556 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:58.172702 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:58.172985 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:58.174435 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:58.175320 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:58.175925 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:58.176042 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:58.176786 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:58.176791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:58.176792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:58.177060 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:58.177105 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:58.177123 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:58.177224 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:58.177455 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 13:07:58.177482 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260926 13:07:58.178329 23454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:58.178687 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:58.180980 23454 catalog_manager.cc:1347] Generated new cluster ID: 1f88505471c44e7490529293395a2752
I20260926 13:07:58.181033 23454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:58.183234 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:58.179989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:52551 (local address 127.0.0.1:45193)
0926 13:07:58.180023 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:58.180025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:58.180210 (+ 185us) server_negotiation.cc:407] Connection header received
0926 13:07:58.180273 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:58.180273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:58.180283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:58.180345 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:58.180637 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:58.181256 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:58.181979 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:07:58.182200 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:07:58.182270 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:07:58.182983 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:58.182985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:58.182985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:58.182989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:58.183003 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:58.183006 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:58.183059 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:58.183127 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 13:07:58.183135 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:07:58.209612 23454 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:58.210448 23454 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:58.215230 23454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09d67fe2c4f74f379c684c9424739d81: Generated new TSK 0
I20260926 13:07:58.215462 23454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:58.223356 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23372
W20260926 13:07:58.227583 23357 connection.cc:535] client connection to 127.0.0.1:45193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45193): Connection reset by peer (error 104)
2026-09-26T13:07:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:58 dist-test-slave-j4lq krb5kdc[23471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:58 dist-test-slave-j4lq krb5kdc[23471](info): set up 2 sockets
Sep 26 13:07:58 dist-test-slave-j4lq krb5kdc[23471](info): commencing operation
krb5kdc: starting...
W20260926 13:07:59.248831 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:07:59 dist-test-slave-j4lq krb5kdc[23471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:07:59.365770 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:07:59.452961 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:07:59.453332 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:07:59.453416 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:07:59.455370 23484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:07:59.455438 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:07:59.455482 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:07:59.455556 23484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:07:59.455586 23484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:07:59.457792 23484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:07:59.458065 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:07:59.458343 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:07:59.465114 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:07:59.465144 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:59.465310 23484 server_base.cc:1034] running on GCE node
W20260926 13:07:59.465124 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:07:59.465762 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:07:59.466120 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:07:59.467397 23484 hybrid_clock.cc:648] HybridClock initialized: now 1790428079467379 us; error 41 us; skew 500 ppm
Sep 26 13:07:59 dist-test-slave-j4lq krb5kdc[23471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428079, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:07:59.470553 23484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:07:59.470968 23484 webserver.cc:460] Webserver started at http://127.0.0.1:43883/ using document root <none> and password file <none>
I20260926 13:07:59.471206 23484 fs_manager.cc:359] Metadata directory not provided
I20260926 13:07:59.471272 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:07:59.471411 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:07:59.472438 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "575edd17839d4cb2a82110d1fea72149"
format_stamp: "Formatted at 2026-09-26 13:07:59 on dist-test-slave-j4lq"
server_key: "a34a7ba37e6151eb7b3c315ae2333181"
server_key_iv: "d0fb4cdadc3d7c5cf945aa0803e1b686"
server_key_version: "encryptionkey@0"
I20260926 13:07:59.472795 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "575edd17839d4cb2a82110d1fea72149"
format_stamp: "Formatted at 2026-09-26 13:07:59 on dist-test-slave-j4lq"
server_key: "a34a7ba37e6151eb7b3c315ae2333181"
server_key_iv: "d0fb4cdadc3d7c5cf945aa0803e1b686"
server_key_version: "encryptionkey@0"
I20260926 13:07:59.474190 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:07:59.474913 23499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:07:59.475075 23484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:07:59.475170 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "575edd17839d4cb2a82110d1fea72149"
format_stamp: "Formatted at 2026-09-26 13:07:59 on dist-test-slave-j4lq"
server_key: "a34a7ba37e6151eb7b3c315ae2333181"
server_key_iv: "d0fb4cdadc3d7c5cf945aa0803e1b686"
server_key_version: "encryptionkey@0"
I20260926 13:07:59.475239 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:07:59.478404 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:07:59.480571 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:07:59.480726 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:07:59.484530 23484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42443
I20260926 13:07:59.484584 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42443 every 8 connection(s)
I20260926 13:07:59.484958 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:07:59.485940 23552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:07:59.488085 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149: Bootstrap starting.
I20260926 13:07:59.488636 23552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149: Neither blocks nor log segments found. Creating new log.
I20260926 13:07:59.488962 23552 log.cc:824] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149: Log is configured to *not* fsync() on all Append() calls
I20260926 13:07:59.489593 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149: No bootstrap required, opened a new log
I20260926 13:07:59.490334 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23484
I20260926 13:07:59.490482 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:07:59.490679 2392 external_mini_cluster.cc:1428] Setting key 89605189544b7bc151161b70c8191bab
I20260926 13:07:59.491573 23552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260926 13:07:59.491744 23552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:07:59.491778 23552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575edd17839d4cb2a82110d1fea72149, State: Initialized, Role: FOLLOWER
I20260926 13:07:59.491947 23552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260926 13:07:59.492110 23552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:07:59.492165 23552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:07:59.492218 23552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:07:59.493084 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:59.491374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:52239 (local address 127.0.0.1:42443)
0926 13:07:59.491560 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:59.491570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:59.491601 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:07:59.491710 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:59.491716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:59.491745 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:59.491912 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:59.492582 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:59.492585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:59.492586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:59.492589 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:59.492625 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:59.492636 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:59.492693 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:59.492789 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 13:07:59.492821 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260926 13:07:59.493359 23552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260926 13:07:59.493471 23552 leader_election.cc:302] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575edd17839d4cb2a82110d1fea72149; no voters:
I20260926 13:07:59.493637 23552 leader_election.cc:288] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:07:59.493736 23557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:07:59.493860 23552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:07:59.493959 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:07:59.493920 23557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [term 1 LEADER]: Becoming Leader. State: Replica: 575edd17839d4cb2a82110d1fea72149, State: Running, Role: LEADER
I20260926 13:07:59.495393 23557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260926 13:07:59.495764 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575edd17839d4cb2a82110d1fea72149. Latest consensus state: current_term: 1 leader_uuid: "575edd17839d4cb2a82110d1fea72149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260926 13:07:59.495843 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:59.495952 23558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575edd17839d4cb2a82110d1fea72149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575edd17839d4cb2a82110d1fea72149" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260926 13:07:59.496073 23577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:07:59.496222 23558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149 [sys.catalog]: This master's current role is: LEADER
I20260926 13:07:59.496562 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:07:59.495200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:38993 (local address 127.0.0.1:42443)
0926 13:07:59.495255 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:07:59.495257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:07:59.495442 (+ 185us) server_negotiation.cc:407] Connection header received
0926 13:07:59.495465 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:07:59.495465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:07:59.495471 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:07:59.495524 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:07:59.496318 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:07:59.496320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:07:59.496320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:07:59.496326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:07:59.496340 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:07:59.496344 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:07:59.496395 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:07:59.496476 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 13:07:59.496490 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 13:07:59.496649 23577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:07:59.498073 23577 catalog_manager.cc:1347] Generated new cluster ID: 6f0ba6f48f8e4b5b9622e1fb437d5968
I20260926 13:07:59.498126 23577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:07:59.506069 23577 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:07:59.506207 23577 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:07:59.510345 23577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 575edd17839d4cb2a82110d1fea72149: Generated new TSK 0
I20260926 13:07:59.510495 23577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:07:59.512820 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23484
2026-09-26T13:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:07:59 dist-test-slave-j4lq krb5kdc[23583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:07:59 dist-test-slave-j4lq krb5kdc[23583](info): set up 2 sockets
Sep 26 13:07:59 dist-test-slave-j4lq krb5kdc[23583](info): commencing operation
krb5kdc: starting...
W20260926 13:08:00.548977 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:08:00.661643 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:00.741787 23596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:00.742030 23596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:00.742070 23596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:00.743216 23596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:00.743279 23596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:00.743304 23596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:00.743330 23596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:00.743359 23596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:00.744812 23596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:00.745097 23596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:00.745337 23596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:00.750161 23601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:00.750193 23602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:00.750177 23604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:00.750216 23596 server_base.cc:1034] running on GCE node
I20260926 13:08:00.750519 23596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:00.750764 23596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:00.751904 23596 hybrid_clock.cc:648] HybridClock initialized: now 1790428080751890 us; error 29 us; skew 500 ppm
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428080, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:00.754375 23596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:08:00.754801 23596 webserver.cc:460] Webserver started at http://127.0.0.1:45383/ using document root <none> and password file <none>
I20260926 13:08:00.755002 23596 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:00.755043 23596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:00.755128 23596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:00.756203 23596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e"
format_stamp: "Formatted at 2026-09-26 13:08:00 on dist-test-slave-j4lq"
server_key: "7617bee722755769371bbf56f6b291d0"
server_key_iv: "f6451c91385eb8002fd97ffefa0c2e9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:00.756507 23596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e"
format_stamp: "Formatted at 2026-09-26 13:08:00 on dist-test-slave-j4lq"
server_key: "7617bee722755769371bbf56f6b291d0"
server_key_iv: "f6451c91385eb8002fd97ffefa0c2e9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:00.757844 23596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:08:00.758544 23611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:00.758710 23596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:00.758798 23596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e"
format_stamp: "Formatted at 2026-09-26 13:08:00 on dist-test-slave-j4lq"
server_key: "7617bee722755769371bbf56f6b291d0"
server_key_iv: "f6451c91385eb8002fd97ffefa0c2e9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:00.758874 23596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:00.769186 23596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:00.771348 23596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:00.771524 23596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:00.775452 23596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38709
I20260926 13:08:00.775483 23663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38709 every 8 connection(s)
I20260926 13:08:00.775863 23596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:00.776134 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23596
I20260926 13:08:00.776261 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:00.776455 2392 external_mini_cluster.cc:1428] Setting key 5c3d94cd085f7d431d31957cdc98bbfa
I20260926 13:08:00.777127 23664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:00.780249 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e: Bootstrap starting.
I20260926 13:08:00.780915 23664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:00.781220 23664 log.cc:824] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:00.781899 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e: No bootstrap required, opened a new log
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 13:08:00.783727 23664 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260926 13:08:00.783879 23664 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:00.783928 23664 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b1b6ac3bcc437f86be32c4ddaaf99e, State: Initialized, Role: FOLLOWER
I20260926 13:08:00.784060 23664 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260926 13:08:00.784132 23664 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:00.784154 23664 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:00.784214 23664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:00.784927 23664 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260926 13:08:00.785041 23664 leader_election.cc:302] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b1b6ac3bcc437f86be32c4ddaaf99e; no voters:
I20260926 13:08:00.785238 23664 leader_election.cc:288] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:00.785287 23669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:00.785475 23669 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [term 1 LEADER]: Becoming Leader. State: Replica: c0b1b6ac3bcc437f86be32c4ddaaf99e, State: Running, Role: LEADER
I20260926 13:08:00.785523 23664 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:00.785629 23669 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260926 13:08:00.786108 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } } }
I20260926 13:08:00.786131 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b1b6ac3bcc437f86be32c4ddaaf99e. Latest consensus state: current_term: 1 leader_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b1b6ac3bcc437f86be32c4ddaaf99e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } } }
I20260926 13:08:00.786275 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:00.786286 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:00.787688 23678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:00.788018 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:00.777004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:46239 (local address 127.0.0.1:38709)
0926 13:08:00.777218 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:00.777229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:00.777264 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:08:00.777829 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:00.777839 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:00.777910 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:00.778062 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:00.778645 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:00.779996 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:00.780798 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:00.781458 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:00.781587 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:00.783919 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:00.783923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:00.783924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:00.783936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:00.785986 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:00.786504 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:00.786507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:00.786507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:00.786629 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:00.786956 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:00.786956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:00.786957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:00.787240 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:00.787620 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:00.787714 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:08:00.787814 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260926 13:08:00.788185 23678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:00.788996 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:00.790313 23678 catalog_manager.cc:1347] Generated new cluster ID: 307b1f1648dc447a8e0a3e20a04a7330
I20260926 13:08:00.790367 23678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:00.797887 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:00.790484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:58451 (local address 127.0.0.1:38709)
0926 13:08:00.790519 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:00.790521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:00.790734 (+ 213us) server_negotiation.cc:407] Connection header received
0926 13:08:00.790792 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:00.790793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:00.790802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:00.790853 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:00.791471 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:00.792177 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:00.793296 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:00.793625 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:00.793683 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:00.795079 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:00.795080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:00.795080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:00.795086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:00.796631 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:00.797096 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:00.797097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:00.797097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:00.797150 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:00.797382 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:00.797382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:00.797383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:00.797495 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:00.797591 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:00.797740 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:08:00.797799 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:08:00.801139 23678 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:00.801630 23678 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:00.812088 23678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b1b6ac3bcc437f86be32c4ddaaf99e: Generated new TSK 0
I20260926 13:08:00.812292 23678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:00.818809 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23596
W20260926 13:08:00.823051 23581 connection.cc:535] client connection to 127.0.0.1:38709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38709): Connection reset by peer (error 104)
2026-09-26T13:08:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23695](info): set up 2 sockets
Sep 26 13:08:00 dist-test-slave-j4lq krb5kdc[23695](info): commencing operation
krb5kdc: starting...
W20260926 13:08:01.853103 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:01 dist-test-slave-j4lq krb5kdc[23695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:08:01.979923 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:02.060684 23708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:02.061033 23708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:02.061066 23708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:02.062340 23708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:02.062383 23708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:02.062398 23708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:02.062410 23708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:02.062422 23708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:02.063910 23708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:02.064126 23708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:02.064311 23708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:02.069083 23713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:02.069072 23714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:02.069072 23716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:02.069098 23708 server_base.cc:1034] running on GCE node
I20260926 13:08:02.069533 23708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:02.069833 23708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:02.070972 23708 hybrid_clock.cc:648] HybridClock initialized: now 1790428082070954 us; error 36 us; skew 500 ppm
Sep 26 13:08:02 dist-test-slave-j4lq krb5kdc[23695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428082, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:02.073475 23708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:08:02.073931 23708 webserver.cc:460] Webserver started at http://127.0.0.1:37477/ using document root <none> and password file <none>
I20260926 13:08:02.074172 23708 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:02.074219 23708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:02.074306 23708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:02.075294 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "faac14c925fd4fcf96edadc749760293"
format_stamp: "Formatted at 2026-09-26 13:08:02 on dist-test-slave-j4lq"
server_key: "b3852ee7dc9faffcc838d74550c58808"
server_key_iv: "56e2fac136caa03d59a090f33fe48719"
server_key_version: "encryptionkey@0"
I20260926 13:08:02.075580 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "faac14c925fd4fcf96edadc749760293"
format_stamp: "Formatted at 2026-09-26 13:08:02 on dist-test-slave-j4lq"
server_key: "b3852ee7dc9faffcc838d74550c58808"
server_key_iv: "56e2fac136caa03d59a090f33fe48719"
server_key_version: "encryptionkey@0"
I20260926 13:08:02.076776 23708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:08:02.077484 23723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:02.077653 23708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:02.077720 23708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "faac14c925fd4fcf96edadc749760293"
format_stamp: "Formatted at 2026-09-26 13:08:02 on dist-test-slave-j4lq"
server_key: "b3852ee7dc9faffcc838d74550c58808"
server_key_iv: "56e2fac136caa03d59a090f33fe48719"
server_key_version: "encryptionkey@0"
I20260926 13:08:02.077801 23708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:02.096740 23708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:02.099025 23708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:02.099186 23708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:02.103315 23708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34551
I20260926 13:08:02.103363 23775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34551 every 8 connection(s)
I20260926 13:08:02.103715 23708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:02.104507 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23708
I20260926 13:08:02.104595 23776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:02.104601 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:02.104837 2392 external_mini_cluster.cc:1428] Setting key 99af04cdf6b585d6e212fd6f7aefa222
I20260926 13:08:02.107425 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293: Bootstrap starting.
I20260926 13:08:02.108016 23776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:02.108354 23776 log.cc:824] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:02.109112 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293: No bootstrap required, opened a new log
I20260926 13:08:02.110319 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:02.105372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:46029 (local address 127.0.0.1:34551)
0926 13:08:02.105552 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:02.105559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:02.105584 (+ 25us) server_negotiation.cc:407] Connection header received
0926 13:08:02.106066 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:02.106072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:02.106123 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:02.106270 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:02.106488 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:02.107361 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:02.108190 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:02.108794 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:02.108934 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:02.109642 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:02.109646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:02.109647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:02.109859 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:02.109896 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:02.109914 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:02.110006 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:02.110085 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 13:08:02.110112 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260926 13:08:02.111339 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:02.111341 23776 raft_consensus.cc:348] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } }
I20260926 13:08:02.111505 23776 raft_consensus.cc:374] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:02.111554 23776 raft_consensus.cc:729] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faac14c925fd4fcf96edadc749760293, State: Initialized, Role: FOLLOWER
I20260926 13:08:02.111673 23776 consensus_queue.cc:260] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } }
I20260926 13:08:02.111745 23776 raft_consensus.cc:388] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:02.111783 23776 raft_consensus.cc:482] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:02.111829 23776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:02.112352 23776 raft_consensus.cc:504] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } }
I20260926 13:08:02.112466 23776 leader_election.cc:302] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faac14c925fd4fcf96edadc749760293; no voters:
I20260926 13:08:02.112655 23776 leader_election.cc:288] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:02.112690 23785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:02.112926 23785 raft_consensus.cc:686] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [term 1 LEADER]: Becoming Leader. State: Replica: faac14c925fd4fcf96edadc749760293, State: Running, Role: LEADER
I20260926 13:08:02.112998 23776 sys_catalog.cc:564] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:02.113057 23785 consensus_queue.cc:237] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } }
I20260926 13:08:02.113423 23785 sys_catalog.cc:455] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faac14c925fd4fcf96edadc749760293. Latest consensus state: current_term: 1 leader_uuid: "faac14c925fd4fcf96edadc749760293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } } }
I20260926 13:08:02.113436 23787 sys_catalog.cc:455] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faac14c925fd4fcf96edadc749760293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faac14c925fd4fcf96edadc749760293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34551 } } }
I20260926 13:08:02.113595 23787 sys_catalog.cc:458] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:02.113816 23785 sys_catalog.cc:458] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:02.114230 23795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:02.114753 23795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:02.116259 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:02.112748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:60813 (local address 127.0.0.1:34551)
0926 13:08:02.112802 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:02.112804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:02.113004 (+ 200us) server_negotiation.cc:407] Connection header received
0926 13:08:02.113115 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:02.113116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:02.113125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:02.113174 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:02.113315 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:02.114042 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:02.114852 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:02.115155 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:02.115238 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:02.115893 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:02.115894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:02.115895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:02.115899 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:02.115912 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:02.115916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:02.115961 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:02.116162 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 13:08:02.116178 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 13:08:02.116784 23795 catalog_manager.cc:1347] Generated new cluster ID: e7844fe6ce0e4dd29206ce6eb1f0489b
I20260926 13:08:02.116823 23795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:02.132340 23795 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:02.133095 23795 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:02.141129 23795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faac14c925fd4fcf96edadc749760293: Generated new TSK 0
I20260926 13:08:02.141345 23795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:02.146160 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23708
W20260926 13:08:02.150452 23693 connection.cc:535] client connection to 127.0.0.1:34551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34551): Connection reset by peer (error 104)
2026-09-26T13:08:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:02 dist-test-slave-j4lq krb5kdc[23807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:02 dist-test-slave-j4lq krb5kdc[23807](info): set up 2 sockets
Sep 26 13:08:02 dist-test-slave-j4lq krb5kdc[23807](info): commencing operation
krb5kdc: starting...
W20260926 13:08:03.176919 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:03 dist-test-slave-j4lq krb5kdc[23807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 13:08:03.296404 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:03.377158 23820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:03.377372 23820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:03.377395 23820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:03.378556 23820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:03.378599 23820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:03.378613 23820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:03.378629 23820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:03.378641 23820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:03.380110 23820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:03.380357 23820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:03.380641 23820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:03.385311 23828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:03.385352 23825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:03.385351 23826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:03.385636 23820 server_base.cc:1034] running on GCE node
I20260926 13:08:03.385844 23820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:03.386076 23820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:03.387214 23820 hybrid_clock.cc:648] HybridClock initialized: now 1790428083387198 us; error 26 us; skew 500 ppm
Sep 26 13:08:03 dist-test-slave-j4lq krb5kdc[23807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428083, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:03.389683 23820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 13:08:03.390080 23820 webserver.cc:460] Webserver started at http://127.0.0.1:44311/ using document root <none> and password file <none>
I20260926 13:08:03.390293 23820 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:03.390358 23820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:03.390498 23820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:03.391535 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "2231a1a614d64ea89c0f70eeb54cc467"
format_stamp: "Formatted at 2026-09-26 13:08:03 on dist-test-slave-j4lq"
server_key: "4056595eb20df85cc2313ab0953685a6"
server_key_iv: "f034ee2caaa09a0354601460376ad23e"
server_key_version: "encryptionkey@0"
I20260926 13:08:03.391896 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "2231a1a614d64ea89c0f70eeb54cc467"
format_stamp: "Formatted at 2026-09-26 13:08:03 on dist-test-slave-j4lq"
server_key: "4056595eb20df85cc2313ab0953685a6"
server_key_iv: "f034ee2caaa09a0354601460376ad23e"
server_key_version: "encryptionkey@0"
I20260926 13:08:03.393255 23820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:08:03.393919 23835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:03.394085 23820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:03.394178 23820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "2231a1a614d64ea89c0f70eeb54cc467"
format_stamp: "Formatted at 2026-09-26 13:08:03 on dist-test-slave-j4lq"
server_key: "4056595eb20df85cc2313ab0953685a6"
server_key_iv: "f034ee2caaa09a0354601460376ad23e"
server_key_version: "encryptionkey@0"
I20260926 13:08:03.394251 23820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:03.397423 23820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:03.399492 23820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:03.399593 23820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:03.403200 23820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35611
I20260926 13:08:03.403273 23887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35611 every 8 connection(s)
I20260926 13:08:03.403537 23820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:03.404454 23888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:03.406635 23888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467: Bootstrap starting.
I20260926 13:08:03.407183 23888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:03.407449 23888 log.cc:824] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:03.408125 23888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467: No bootstrap required, opened a new log
I20260926 13:08:03.409776 23888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } }
I20260926 13:08:03.409927 23888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:03.409972 23888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2231a1a614d64ea89c0f70eeb54cc467, State: Initialized, Role: FOLLOWER
I20260926 13:08:03.410112 23888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } }
I20260926 13:08:03.410192 23888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:03.410233 23888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:03.410281 23888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:03.410805 23888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } }
I20260926 13:08:03.410918 23888 leader_election.cc:302] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2231a1a614d64ea89c0f70eeb54cc467; no voters:
I20260926 13:08:03.411037 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23820
I20260926 13:08:03.411101 23888 leader_election.cc:288] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:03.411134 23891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:03.411180 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:03.411314 23891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [term 1 LEADER]: Becoming Leader. State: Replica: 2231a1a614d64ea89c0f70eeb54cc467, State: Running, Role: LEADER
I20260926 13:08:03.411387 2392 external_mini_cluster.cc:1428] Setting key 6a7c73749827d276e81b109abf1caf8c
I20260926 13:08:03.411412 23888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:03.411435 23891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } }
I20260926 13:08:03.411749 23892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2231a1a614d64ea89c0f70eeb54cc467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } } }
I20260926 13:08:03.411827 23892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:03.412077 23893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2231a1a614d64ea89c0f70eeb54cc467. Latest consensus state: current_term: 1 leader_uuid: "2231a1a614d64ea89c0f70eeb54cc467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2231a1a614d64ea89c0f70eeb54cc467" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35611 } } }
I20260926 13:08:03.412160 23893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:03.412284 23898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:03.412837 23898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:03.414840 23898 catalog_manager.cc:1347] Generated new cluster ID: 6520b6614d4c411e85d269d5dabf7c12
I20260926 13:08:03.414902 23898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:03.415024 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:03.412016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:42579 (local address 127.0.0.1:35611)
0926 13:08:03.412204 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:03.412214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:03.412244 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:08:03.412365 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:03.412372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:03.412407 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:03.412563 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:03.414561 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:03.414564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:03.414565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:03.414569 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:03.414612 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:03.414623 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:03.414696 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:03.414767 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:08:03.414812 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":56,"threads_started":1}
I20260926 13:08:03.415839 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:03.421984 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:03.417182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.1.8:57523 (local address 127.0.0.1:35611)
0926 13:08:03.417220 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:03.417222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:03.421023 (+ 3801us) server_negotiation.cc:407] Connection header received
0926 13:08:03.421098 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:03.421099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:03.421104 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:03.421154 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:03.421752 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:03.421753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:03.421753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:03.421758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:03.421772 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:03.421776 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:03.421822 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:03.421876 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 13:08:03.421891 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:08:03.461392 23898 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:03.461607 23898 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:03.468673 23898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2231a1a614d64ea89c0f70eeb54cc467: Generated new TSK 0
I20260926 13:08:03.468919 23898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:03.475317 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23820
2026-09-26T13:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:03 dist-test-slave-j4lq krb5kdc[23919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:03 dist-test-slave-j4lq krb5kdc[23919](info): set up 2 sockets
Sep 26 13:08:03 dist-test-slave-j4lq krb5kdc[23919](info): commencing operation
krb5kdc: starting...
W20260926 13:08:04.497076 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:04.607882 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46693
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:44023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:46693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:04.687143 23932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:04.687386 23932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:04.687425 23932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:04.688558 23932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:04.688624 23932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:04.688658 23932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:04.688701 23932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:04.688731 23932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:04.690253 23932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:44023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:46693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:46693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:04.690517 23932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:04.690742 23932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:04.695529 23937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:04.695698 23932 server_base.cc:1034] running on GCE node
W20260926 13:08:04.695524 23940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:04.695550 23938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:04.696034 23932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:04.696318 23932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:04.697465 23932 hybrid_clock.cc:648] HybridClock initialized: now 1790428084697444 us; error 35 us; skew 500 ppm
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[23919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428084, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:04.704108 23932 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:04.704566 23932 webserver.cc:460] Webserver started at http://127.2.86.62:37523/ using document root <none> and password file <none>
I20260926 13:08:04.704807 23932 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:04.704854 23932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:04.705025 23932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:04.706120 23932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "81340fdc997d44a3b9af9bb289465093"
format_stamp: "Formatted at 2026-09-26 13:08:04 on dist-test-slave-j4lq"
server_key: "86ae8c42fe6bcb5541f24bd8747846d2"
server_key_iv: "93ebac2b1df9607ef795cf1422b8a6fb"
server_key_version: "encryptionkey@0"
I20260926 13:08:04.706486 23932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "81340fdc997d44a3b9af9bb289465093"
format_stamp: "Formatted at 2026-09-26 13:08:04 on dist-test-slave-j4lq"
server_key: "86ae8c42fe6bcb5541f24bd8747846d2"
server_key_iv: "93ebac2b1df9607ef795cf1422b8a6fb"
server_key_version: "encryptionkey@0"
I20260926 13:08:04.707844 23932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:08:04.708621 23947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:04.708794 23932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:04.708904 23932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "81340fdc997d44a3b9af9bb289465093"
format_stamp: "Formatted at 2026-09-26 13:08:04 on dist-test-slave-j4lq"
server_key: "86ae8c42fe6bcb5541f24bd8747846d2"
server_key_iv: "93ebac2b1df9607ef795cf1422b8a6fb"
server_key_version: "encryptionkey@0"
I20260926 13:08:04.708982 23932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:04.721746 23932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:04.723693 23932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:04.723862 23932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:04.727811 23932 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:46693
I20260926 13:08:04.727860 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:46693 every 8 connection(s)
I20260926 13:08:04.728230 23932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:04.729092 24000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:04.731338 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093: Bootstrap starting.
I20260926 13:08:04.731899 24000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:04.732179 24000 log.cc:824] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:04.732527 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 23932
I20260926 13:08:04.732661 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:04.732843 2392 external_mini_cluster.cc:1428] Setting key ac84a668d441e17f6bd861f25e526cf8
I20260926 13:08:04.732985 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093: No bootstrap required, opened a new log
I20260926 13:08:04.735173 24000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } }
I20260926 13:08:04.735351 24000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:04.735397 24000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81340fdc997d44a3b9af9bb289465093, State: Initialized, Role: FOLLOWER
I20260926 13:08:04.735559 24000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } }
I20260926 13:08:04.735636 24000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:04.735666 24000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:04.735709 24000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:04.736400 24000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } }
I20260926 13:08:04.736518 24000 leader_election.cc:302] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81340fdc997d44a3b9af9bb289465093; no voters:
I20260926 13:08:04.736672 24000 leader_election.cc:288] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:04.736719 24005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:04.736910 24005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [term 1 LEADER]: Becoming Leader. State: Replica: 81340fdc997d44a3b9af9bb289465093, State: Running, Role: LEADER
I20260926 13:08:04.736955 24000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:04.737013 24005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } }
I20260926 13:08:04.737392 24005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81340fdc997d44a3b9af9bb289465093. Latest consensus state: current_term: 1 leader_uuid: "81340fdc997d44a3b9af9bb289465093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } } }
I20260926 13:08:04.737602 24005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:04.737535 24006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81340fdc997d44a3b9af9bb289465093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81340fdc997d44a3b9af9bb289465093" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 46693 } } }
I20260926 13:08:04.737838 24006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:04.738214 24014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:04.738724 24014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:04.740798 24014 catalog_manager.cc:1347] Generated new cluster ID: 8fabd8793f454f529f23c18ecdc2708d
I20260926 13:08:04.740854 24014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[23919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:04.744922 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:04.733364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50833 (local address 127.2.86.62:46693)
0926 13:08:04.733558 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:04.733568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:04.733604 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:08:04.734365 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:04.734374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:04.734438 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:04.734581 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:04.735150 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:04.736901 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:04.738185 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:04.738843 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:04.738948 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:04.741619 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:04.741624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:04.741624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:04.741637 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:04.743302 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:04.743691 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:04.743693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:04.743694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:04.743771 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:04.743993 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:04.743994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:04.743995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:04.744222 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:04.744401 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:04.744486 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 13:08:04.744586 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260926 13:08:04.745879 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:04.753582 24014 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:04.754338 24014 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:04.755012 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:04.747162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32845 (local address 127.2.86.62:46693)
0926 13:08:04.747199 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:04.747201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:04.749050 (+ 1849us) server_negotiation.cc:407] Connection header received
0926 13:08:04.749122 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:04.749123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:04.749132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:04.749187 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:04.749703 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:04.750194 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:04.750877 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:04.751103 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:04.751228 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:04.752334 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:04.752335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:04.752335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:04.752342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:04.753864 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:04.754204 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:04.754205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:04.754205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:04.754255 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:04.754490 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:04.754490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:04.754491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:04.754592 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:04.754699 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:04.754838 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 13:08:04.754900 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:08:04.760691 24014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81340fdc997d44a3b9af9bb289465093: Generated new TSK 0
I20260926 13:08:04.760892 24014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:04.763976 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 23932
W20260926 13:08:04.768471 23917 connection.cc:535] client connection to 127.2.86.62:46693 recv error: Network error: failed to read from TLS socket (remote: 127.2.86.62:46693): Connection reset by peer (error 104)
2026-09-26T13:08:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[24031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[24031](info): set up 2 sockets
Sep 26 13:08:04 dist-test-slave-j4lq krb5kdc[24031](info): commencing operation
krb5kdc: starting...
W20260926 13:08:05.788832 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:05 dist-test-slave-j4lq krb5kdc[24031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:05.908375 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41275
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:41275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:05.987835 24044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:05.988075 24044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:05.988113 24044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:05.989332 24044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:05.989393 24044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:05.989419 24044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:05.989452 24044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:05.989476 24044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:05.990942 24044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:41275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:41275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:05.991230 24044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:05.991454 24044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:05.996260 24049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:05.996316 24050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:05.996428 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:05.996402 24044 server_base.cc:1034] running on GCE node
I20260926 13:08:05.996706 24044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:05.997043 24044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:05.998214 24044 hybrid_clock.cc:648] HybridClock initialized: now 1790428085998190 us; error 39 us; skew 500 ppm
Sep 26 13:08:05 dist-test-slave-j4lq krb5kdc[24031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428085, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:06.000835 24044 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:06.001351 24044 webserver.cc:460] Webserver started at http://127.2.86.62:36115/ using document root <none> and password file <none>
I20260926 13:08:06.001585 24044 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:06.001653 24044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:06.001818 24044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:06.002871 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "3df385040cdf4cc7b69ce2946f9e30f4"
format_stamp: "Formatted at 2026-09-26 13:08:06 on dist-test-slave-j4lq"
server_key: "04262ae015bfb6b636ef3a8b2cd8a85a"
server_key_iv: "426849ac5bf35fd41a17464b44d36794"
server_key_version: "encryptionkey@0"
I20260926 13:08:06.003235 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "3df385040cdf4cc7b69ce2946f9e30f4"
format_stamp: "Formatted at 2026-09-26 13:08:06 on dist-test-slave-j4lq"
server_key: "04262ae015bfb6b636ef3a8b2cd8a85a"
server_key_iv: "426849ac5bf35fd41a17464b44d36794"
server_key_version: "encryptionkey@0"
I20260926 13:08:06.004593 24044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 13:08:06.005420 24059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:06.005625 24044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:08:06.005733 24044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "3df385040cdf4cc7b69ce2946f9e30f4"
format_stamp: "Formatted at 2026-09-26 13:08:06 on dist-test-slave-j4lq"
server_key: "04262ae015bfb6b636ef3a8b2cd8a85a"
server_key_iv: "426849ac5bf35fd41a17464b44d36794"
server_key_version: "encryptionkey@0"
I20260926 13:08:06.005820 24044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:06.014631 24044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:06.016973 24044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:06.017128 24044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:06.020768 24044 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:41275
I20260926 13:08:06.020833 24111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:41275 every 8 connection(s)
I20260926 13:08:06.021205 24044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:06.022008 24112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:06.022753 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24044
I20260926 13:08:06.022888 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:06.023078 2392 external_mini_cluster.cc:1428] Setting key 2e0c00ca3f959c9c1cc510a106f28270
I20260926 13:08:06.025247 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: Bootstrap starting.
I20260926 13:08:06.025915 24112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:06.026202 24112 log.cc:824] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:06.026870 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: No bootstrap required, opened a new log
I20260926 13:08:06.029091 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:06.023605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40643 (local address 127.2.86.62:41275)
0926 13:08:06.023817 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:06.023826 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:06.023861 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:08:06.024380 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:06.024388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:06.024452 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:06.024613 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:06.024780 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:06.026160 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:06.026943 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:06.027539 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:06.027670 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:06.028330 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:06.028334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:06.028335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:06.028569 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:06.028615 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:06.028632 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:06.028737 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:06.028815 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 13:08:06.028845 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260926 13:08:06.029246 24112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } }
I20260926 13:08:06.029369 24112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:06.029422 24112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df385040cdf4cc7b69ce2946f9e30f4, State: Initialized, Role: FOLLOWER
I20260926 13:08:06.029546 24112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } }
I20260926 13:08:06.029629 24112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:06.029686 24112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:06.029742 24112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:06.030063 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:06.030352 24112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } }
I20260926 13:08:06.030453 24112 leader_election.cc:302] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df385040cdf4cc7b69ce2946f9e30f4; no voters:
I20260926 13:08:06.030627 24112 leader_election.cc:288] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:06.030685 24117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:06.030833 24117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [term 1 LEADER]: Becoming Leader. State: Replica: 3df385040cdf4cc7b69ce2946f9e30f4, State: Running, Role: LEADER
I20260926 13:08:06.030864 24112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:06.032310 24117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } }
I20260926 13:08:06.032894 24118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df385040cdf4cc7b69ce2946f9e30f4. Latest consensus state: current_term: 1 leader_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } } }
I20260926 13:08:06.033063 24119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df385040cdf4cc7b69ce2946f9e30f4" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 41275 } } }
I20260926 13:08:06.033248 24118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:06.033255 24119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4 [sys.catalog]: This master's current role is: LEADER
W20260926 13:08:06.033440 24136 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:06.033490 24136 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:06.033547 24139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:06.033726 24139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:06.035022 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:06.031614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60623 (local address 127.2.86.62:41275)
0926 13:08:06.031650 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:06.031652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:06.031751 (+ 99us) server_negotiation.cc:407] Connection header received
0926 13:08:06.031816 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:06.031816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:06.031826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:06.031900 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:06.032117 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:06.032787 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:06.033856 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:06.034048 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:06.034123 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:06.034784 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:06.034785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:06.034786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:06.034790 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:06.034803 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:06.034807 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:06.034858 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:06.034917 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 13:08:06.034926 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:08:06.035113 24139 catalog_manager.cc:1347] Generated new cluster ID: 864c93178ea645d3a1760bd6a23209fd
I20260926 13:08:06.035154 24139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:06.057344 24139 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:06.058089 24139 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:06.066363 24139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3df385040cdf4cc7b69ce2946f9e30f4: Generated new TSK 0
I20260926 13:08:06.066569 24139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:06.079331 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24044
W20260926 13:08:06.083699 24029 connection.cc:535] client connection to 127.2.86.62:41275 recv error: Network error: failed to read from TLS socket (remote: 127.2.86.62:41275): Connection reset by peer (error 104)
2026-09-26T13:08:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:06 dist-test-slave-j4lq krb5kdc[24143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:06 dist-test-slave-j4lq krb5kdc[24143](info): set up 2 sockets
Sep 26 13:08:06 dist-test-slave-j4lq krb5kdc[24143](info): commencing operation
krb5kdc: starting...
W20260926 13:08:07.104924 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:07 dist-test-slave-j4lq krb5kdc[24143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:07.216707 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42463
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:42463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:07.297271 24156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:07.297488 24156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:07.297525 24156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:07.298666 24156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:07.298732 24156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:07.298758 24156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:07.298787 24156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:07.298815 24156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:07.300246 24156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:42463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:07.300509 24156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:07.300726 24156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:07.305508 24164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:07.305529 24161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:07.305562 24162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:07.305711 24156 server_base.cc:1034] running on GCE node
I20260926 13:08:07.305964 24156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:07.306279 24156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:07.307427 24156 hybrid_clock.cc:648] HybridClock initialized: now 1790428087307416 us; error 28 us; skew 500 ppm
Sep 26 13:08:07 dist-test-slave-j4lq krb5kdc[24143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428087, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:07.309965 24156 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:07.310382 24156 webserver.cc:460] Webserver started at http://127.2.86.62:38603/ using document root <none> and password file <none>
I20260926 13:08:07.310647 24156 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:07.310691 24156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:07.310817 24156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:07.311838 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "ea983fd436104e6e9ee1b23e9a05aeea"
format_stamp: "Formatted at 2026-09-26 13:08:07 on dist-test-slave-j4lq"
server_key: "aeee674237e5da01647e83a49c73c1b8"
server_key_iv: "c0e4455a5aaee34c379a520e8529dd11"
server_key_version: "encryptionkey@0"
I20260926 13:08:07.312194 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "ea983fd436104e6e9ee1b23e9a05aeea"
format_stamp: "Formatted at 2026-09-26 13:08:07 on dist-test-slave-j4lq"
server_key: "aeee674237e5da01647e83a49c73c1b8"
server_key_iv: "c0e4455a5aaee34c379a520e8529dd11"
server_key_version: "encryptionkey@0"
I20260926 13:08:07.313572 24156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:08:07.314260 24171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:07.314430 24156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:07.314527 24156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "ea983fd436104e6e9ee1b23e9a05aeea"
format_stamp: "Formatted at 2026-09-26 13:08:07 on dist-test-slave-j4lq"
server_key: "aeee674237e5da01647e83a49c73c1b8"
server_key_iv: "c0e4455a5aaee34c379a520e8529dd11"
server_key_version: "encryptionkey@0"
I20260926 13:08:07.314607 24156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:07.317925 24156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:07.319862 24156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:07.320019 24156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:07.324139 24156 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:42463
I20260926 13:08:07.324191 24223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:42463 every 8 connection(s)
I20260926 13:08:07.324534 24156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:07.325519 24224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:07.327656 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: Bootstrap starting.
I20260926 13:08:07.328217 24224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:07.328490 24224 log.cc:824] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:07.329198 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: No bootstrap required, opened a new log
I20260926 13:08:07.330716 24224 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } }
I20260926 13:08:07.330888 24224 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:07.330927 24224 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea983fd436104e6e9ee1b23e9a05aeea, State: Initialized, Role: FOLLOWER
I20260926 13:08:07.330999 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24156
I20260926 13:08:07.331105 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:07.331085 24224 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } }
I20260926 13:08:07.331173 24224 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:07.331216 24224 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:07.331269 24224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:07.331305 2392 external_mini_cluster.cc:1428] Setting key 84c44d681dcff02b4e54a98eb659eb92
I20260926 13:08:07.332218 24224 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } }
I20260926 13:08:07.332341 24224 leader_election.cc:302] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea983fd436104e6e9ee1b23e9a05aeea; no voters:
I20260926 13:08:07.332530 24224 leader_election.cc:288] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:07.332715 24230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:07.332779 24224 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:07.332842 24230 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [term 1 LEADER]: Becoming Leader. State: Replica: ea983fd436104e6e9ee1b23e9a05aeea, State: Running, Role: LEADER
I20260926 13:08:07.333030 24230 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } }
I20260926 13:08:07.333554 24231 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea983fd436104e6e9ee1b23e9a05aeea. Latest consensus state: current_term: 1 leader_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } } }
I20260926 13:08:07.333626 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:07.331775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41469 (local address 127.2.86.62:42463)
0926 13:08:07.331986 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:07.331996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:07.332026 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:08:07.332143 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:07.332148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:07.332170 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:07.332299 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:07.333096 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:07.333100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:07.333100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:07.333105 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:07.333150 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:07.333165 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:07.333207 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:07.333361 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 13:08:07.333416 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260926 13:08:07.333654 24231 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:07.333562 24229 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea983fd436104e6e9ee1b23e9a05aeea" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42463 } } }
I20260926 13:08:07.333806 24229 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:07.334612 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 13:08:07.334602 24244 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:07.334652 24244 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:07.334800 24238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:07.334926 24238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:07.336596 24238 catalog_manager.cc:1347] Generated new cluster ID: 04ef3208d16c48758940ed9cb5ab029d
I20260926 13:08:07.336651 24238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:07.337512 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:07.336207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46887 (local address 127.2.86.62:42463)
0926 13:08:07.336240 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:07.336242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:07.336406 (+ 164us) server_negotiation.cc:407] Connection header received
0926 13:08:07.336577 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:07.336577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:07.336583 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:07.336636 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:07.337298 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:07.337299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:07.337299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:07.337303 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:07.337317 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:07.337321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:07.337350 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:07.337444 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 13:08:07.337456 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260926 13:08:07.358244 24238 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:07.358438 24238 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:07.367904 24238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea983fd436104e6e9ee1b23e9a05aeea: Generated new TSK 0
I20260926 13:08:07.368108 24238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:07.373692 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24156
2026-09-26T13:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:07 dist-test-slave-j4lq krb5kdc[24255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:07 dist-test-slave-j4lq krb5kdc[24255](info): set up 2 sockets
Sep 26 13:08:07 dist-test-slave-j4lq krb5kdc[24255](info): commencing operation
krb5kdc: starting...
W20260926 13:08:08.405176 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:08.524025 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33983
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:45053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:33983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:08.604125 24268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:08.604375 24268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:08.604414 24268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:08.605703 24268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:08.605764 24268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:08.605788 24268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:08.605814 24268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:08.605844 24268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:08.607334 24268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:45053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:33983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:33983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:08.607626 24268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:08.607865 24268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:08.612690 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:08.612717 24268 server_base.cc:1034] running on GCE node
W20260926 13:08:08.612691 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:08.612723 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:08.613160 24268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:08.613453 24268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:08.614605 24268 hybrid_clock.cc:648] HybridClock initialized: now 1790428088614592 us; error 33 us; skew 500 ppm
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428088, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:08.617141 24268 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:08.617630 24268 webserver.cc:460] Webserver started at http://127.2.86.62:41327/ using document root <none> and password file <none>
I20260926 13:08:08.617874 24268 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:08.617944 24268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:08.618098 24268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:08.619146 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "ad898abca1804d14af61e8ffeb7c95c5"
format_stamp: "Formatted at 2026-09-26 13:08:08 on dist-test-slave-j4lq"
server_key: "118fa7180e86a921faeb61bd048a9eeb"
server_key_iv: "f00ac59ba3d7e563bee4a14a359a7c59"
server_key_version: "encryptionkey@0"
I20260926 13:08:08.619506 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "ad898abca1804d14af61e8ffeb7c95c5"
format_stamp: "Formatted at 2026-09-26 13:08:08 on dist-test-slave-j4lq"
server_key: "118fa7180e86a921faeb61bd048a9eeb"
server_key_iv: "f00ac59ba3d7e563bee4a14a359a7c59"
server_key_version: "encryptionkey@0"
I20260926 13:08:08.620819 24268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:08:08.621573 24283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:08.621743 24268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:08.621846 24268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "ad898abca1804d14af61e8ffeb7c95c5"
format_stamp: "Formatted at 2026-09-26 13:08:08 on dist-test-slave-j4lq"
server_key: "118fa7180e86a921faeb61bd048a9eeb"
server_key_iv: "f00ac59ba3d7e563bee4a14a359a7c59"
server_key_version: "encryptionkey@0"
I20260926 13:08:08.621919 24268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:08.630914 24268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:08.632638 24268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:08.632792 24268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:08.636958 24268 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:33983
I20260926 13:08:08.636979 24335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:33983 every 8 connection(s)
I20260926 13:08:08.637328 24268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:08.638181 24336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:08.638842 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24268
I20260926 13:08:08.639010 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:08.639202 2392 external_mini_cluster.cc:1428] Setting key 3ba58d3224ac830bd0c14b972ea0b4c1
I20260926 13:08:08.641343 24336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5: Bootstrap starting.
I20260926 13:08:08.641930 24336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:08.642237 24336 log.cc:824] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:08.642931 24336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5: No bootstrap required, opened a new log
I20260926 13:08:08.644721 24336 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } }
I20260926 13:08:08.644876 24336 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:08.644974 24336 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad898abca1804d14af61e8ffeb7c95c5, State: Initialized, Role: FOLLOWER
I20260926 13:08:08.645128 24336 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } }
I20260926 13:08:08.645211 24336 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:08.645254 24336 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:08.645303 24336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:08.645819 24336 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } }
I20260926 13:08:08.645932 24336 leader_election.cc:302] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad898abca1804d14af61e8ffeb7c95c5; no voters:
I20260926 13:08:08.646142 24336 leader_election.cc:288] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:08.646183 24341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:08.646296 24341 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [term 1 LEADER]: Becoming Leader. State: Replica: ad898abca1804d14af61e8ffeb7c95c5, State: Running, Role: LEADER
I20260926 13:08:08.646417 24336 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:08.646425 24341 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } }
I20260926 13:08:08.646819 24341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad898abca1804d14af61e8ffeb7c95c5. Latest consensus state: current_term: 1 leader_uuid: "ad898abca1804d14af61e8ffeb7c95c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } } }
I20260926 13:08:08.646968 24341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:08.647030 24342 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad898abca1804d14af61e8ffeb7c95c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad898abca1804d14af61e8ffeb7c95c5" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 33983 } } }
I20260926 13:08:08.647189 24342 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:08.647224 24350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:08.647779 24350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:08.649685 24350 catalog_manager.cc:1347] Generated new cluster ID: d3179414d23948f499f9b93e0baa8059
I20260926 13:08:08.649740 24350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:08.650288 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:08.639734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42081 (local address 127.2.86.62:33983)
0926 13:08:08.639932 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:08.639941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:08.639975 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:08:08.640560 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:08.640569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:08.640635 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:08.640798 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:08.641479 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:08.642479 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:08.643309 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:08.643719 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:08.643840 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:08.646402 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:08.646406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:08.646406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:08.646418 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:08.648337 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:08.648744 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:08.648746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:08.648747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:08.648843 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:08.649093 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:08.649093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:08.649094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:08.649404 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:08.649545 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:08.649972 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 13:08:08.650104 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260926 13:08:08.651249 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:08.659745 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:08.652629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45539 (local address 127.2.86.62:33983)
0926 13:08:08.652662 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:08.652664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:08.652757 (+ 93us) server_negotiation.cc:407] Connection header received
0926 13:08:08.652816 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:08.652816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:08.652824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:08.652895 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:08.653530 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:08.654291 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:08.655304 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:08.655610 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:08.655663 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:08.657080 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:08.657081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:08.657082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:08.657088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:08.658622 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:08.659013 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:08.659014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:08.659015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:08.659074 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:08.659273 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:08.659274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:08.659274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:08.659381 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:08.659461 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:08.659595 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 13:08:08.659661 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 13:08:08.664621 24350 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:08.665179 24350 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:08.668602 24350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad898abca1804d14af61e8ffeb7c95c5: Generated new TSK 0
I20260926 13:08:08.668761 24350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:08.673022 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24268
2026-09-26T13:08:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24367](info): set up 2 sockets
Sep 26 13:08:08 dist-test-slave-j4lq krb5kdc[24367](info): commencing operation
krb5kdc: starting...
W20260926 13:08:09.705099 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:09 dist-test-slave-j4lq krb5kdc[24367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:09.817359 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:36259
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:36259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:09.899343 24380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:09.899540 24380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:09.899570 24380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:09.900653 24380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:09.900687 24380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:09.900701 24380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:09.900713 24380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:09.900725 24380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:09.902172 24380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:36259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:36259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:09.902385 24380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:09.902563 24380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:09.907209 24385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:09.907362 24380 server_base.cc:1034] running on GCE node
W20260926 13:08:09.907169 24388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:09.907200 24386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:09.907727 24380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:09.908051 24380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:09.909209 24380 hybrid_clock.cc:648] HybridClock initialized: now 1790428089909187 us; error 33 us; skew 500 ppm
Sep 26 13:08:09 dist-test-slave-j4lq krb5kdc[24367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:09.915629 24380 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:09.916141 24380 webserver.cc:460] Webserver started at http://127.2.86.62:34585/ using document root <none> and password file <none>
I20260926 13:08:09.916371 24380 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:09.916440 24380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:09.916595 24380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:09.917749 24380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "14f88386225f4367a0c40e85c75e05f7"
format_stamp: "Formatted at 2026-09-26 13:08:09 on dist-test-slave-j4lq"
server_key: "5539ab1ed40daef1b656f97ada2f7966"
server_key_iv: "2d2f7caf5d442c8153b60ede4fa4ec9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:09.918130 24380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "14f88386225f4367a0c40e85c75e05f7"
format_stamp: "Formatted at 2026-09-26 13:08:09 on dist-test-slave-j4lq"
server_key: "5539ab1ed40daef1b656f97ada2f7966"
server_key_iv: "2d2f7caf5d442c8153b60ede4fa4ec9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:09.919472 24380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:08:09.920204 24395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:09.920375 24380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:09.920486 24380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "14f88386225f4367a0c40e85c75e05f7"
format_stamp: "Formatted at 2026-09-26 13:08:09 on dist-test-slave-j4lq"
server_key: "5539ab1ed40daef1b656f97ada2f7966"
server_key_iv: "2d2f7caf5d442c8153b60ede4fa4ec9f"
server_key_version: "encryptionkey@0"
I20260926 13:08:09.920567 24380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:09.935487 24380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:09.937893 24380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:09.938066 24380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:09.942147 24380 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:36259
I20260926 13:08:09.942206 24447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:36259 every 8 connection(s)
I20260926 13:08:09.942574 24380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:09.943394 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:09.945703 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7: Bootstrap starting.
I20260926 13:08:09.946288 24448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:09.946578 24448 log.cc:824] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:09.947296 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7: No bootstrap required, opened a new log
I20260926 13:08:09.948843 24448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } }
I20260926 13:08:09.949007 24448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:09.949030 24448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f88386225f4367a0c40e85c75e05f7, State: Initialized, Role: FOLLOWER
I20260926 13:08:09.949223 24448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } }
I20260926 13:08:09.949302 24448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:09.949342 24448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:09.949391 24448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:09.950214 24448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } }
I20260926 13:08:09.950323 24448 leader_election.cc:302] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14f88386225f4367a0c40e85c75e05f7; no voters:
I20260926 13:08:09.950500 24448 leader_election.cc:288] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:09.950562 24451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:09.950728 24451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [term 1 LEADER]: Becoming Leader. State: Replica: 14f88386225f4367a0c40e85c75e05f7, State: Running, Role: LEADER
I20260926 13:08:09.950749 24448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:09.950837 24451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } }
I20260926 13:08:09.951287 24452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14f88386225f4367a0c40e85c75e05f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } } }
I20260926 13:08:09.951300 24453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14f88386225f4367a0c40e85c75e05f7. Latest consensus state: current_term: 1 leader_uuid: "14f88386225f4367a0c40e85c75e05f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f88386225f4367a0c40e85c75e05f7" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 36259 } } }
I20260926 13:08:09.951395 24452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:09.951401 24453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:09.951678 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24380
I20260926 13:08:09.951793 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:09.951920 24460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:09.952011 2392 external_mini_cluster.cc:1428] Setting key 7f138134fe2784db9c7cd350f005534c
I20260926 13:08:09.952462 24460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:09.957468 24460 catalog_manager.cc:1347] Generated new cluster ID: a5c0334a4aa64f6abf29fedf960c6c47
I20260926 13:08:09.957574 24460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:09.957717 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:09.952487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54421 (local address 127.2.86.62:36259)
0926 13:08:09.952686 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:09.952696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:09.953049 (+ 353us) server_negotiation.cc:407] Connection header received
0926 13:08:09.953249 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:09.953257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:09.953314 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:09.953417 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:09.953767 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:09.954695 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:09.955463 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:09.955907 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:09.956025 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:09.956759 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:09.956763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:09.956763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:09.957012 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:09.957054 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:09.957066 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:09.957137 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:09.957220 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 13:08:09.957284 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260926 13:08:09.958585 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:09.963573 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:09.960064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33169 (local address 127.2.86.62:36259)
0926 13:08:09.960101 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:09.960103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:09.960266 (+ 163us) server_negotiation.cc:407] Connection header received
0926 13:08:09.960323 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:09.960324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:09.960335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:09.960386 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:09.960527 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:09.961252 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:09.962292 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:09.962579 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:09.962638 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:09.963329 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:09.963330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:09.963330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:09.963336 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:09.963349 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:09.963353 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:09.963403 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:09.963477 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 13:08:09.963498 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:08:10.012658 24460 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:10.013226 24460 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:10.015923 24460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14f88386225f4367a0c40e85c75e05f7: Generated new TSK 0
I20260926 13:08:10.016095 24460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:10.024196 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24380
2026-09-26T13:08:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:10 dist-test-slave-j4lq krb5kdc[24479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:10 dist-test-slave-j4lq krb5kdc[24479](info): set up 2 sockets
Sep 26 13:08:10 dist-test-slave-j4lq krb5kdc[24479](info): commencing operation
krb5kdc: starting...
W20260926 13:08:11.048971 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:11 dist-test-slave-j4lq krb5kdc[24479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:11.162822 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43885
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:43885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:11.241307 24492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:11.241640 24492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:11.241683 24492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:11.242869 24492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:11.242944 24492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:11.242967 24492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:11.243000 24492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:11.243031 24492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:11.244491 24492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:43885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:43885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:11.244771 24492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:11.245018 24492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:11.249624 24500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:11.249630 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:11.249651 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:11.249882 24492 server_base.cc:1034] running on GCE node
I20260926 13:08:11.250116 24492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:11.250407 24492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:11.251587 24492 hybrid_clock.cc:648] HybridClock initialized: now 1790428091251575 us; error 25 us; skew 500 ppm
Sep 26 13:08:11 dist-test-slave-j4lq krb5kdc[24479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428091, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:11.254076 24492 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:11.254631 24492 webserver.cc:460] Webserver started at http://127.2.86.62:39259/ using document root <none> and password file <none>
I20260926 13:08:11.254856 24492 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:11.254922 24492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:11.255052 24492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:11.256070 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "98334f719eaf4b2886dfb9fb456a2d55"
format_stamp: "Formatted at 2026-09-26 13:08:11 on dist-test-slave-j4lq"
server_key: "713009f92e7b50bbe2eff7da0bc4c69c"
server_key_iv: "ece5994b6885f12cdc04f2508298b34b"
server_key_version: "encryptionkey@0"
I20260926 13:08:11.256434 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "98334f719eaf4b2886dfb9fb456a2d55"
format_stamp: "Formatted at 2026-09-26 13:08:11 on dist-test-slave-j4lq"
server_key: "713009f92e7b50bbe2eff7da0bc4c69c"
server_key_iv: "ece5994b6885f12cdc04f2508298b34b"
server_key_version: "encryptionkey@0"
I20260926 13:08:11.257827 24492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 13:08:11.258517 24507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:11.258669 24492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:11.258762 24492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "98334f719eaf4b2886dfb9fb456a2d55"
format_stamp: "Formatted at 2026-09-26 13:08:11 on dist-test-slave-j4lq"
server_key: "713009f92e7b50bbe2eff7da0bc4c69c"
server_key_iv: "ece5994b6885f12cdc04f2508298b34b"
server_key_version: "encryptionkey@0"
I20260926 13:08:11.258841 24492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:11.262032 24492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:11.264258 24492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:11.264405 24492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:11.268321 24492 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:43885
I20260926 13:08:11.268348 24559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:43885 every 8 connection(s)
I20260926 13:08:11.268663 24492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:11.269608 24560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:11.271682 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55: Bootstrap starting.
I20260926 13:08:11.272189 24560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:11.272413 24560 log.cc:824] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:11.273098 24560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55: No bootstrap required, opened a new log
I20260926 13:08:11.274580 24560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } }
I20260926 13:08:11.274729 24560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:08:11.274750 24560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98334f719eaf4b2886dfb9fb456a2d55, State: Initialized, Role: FOLLOWER
I20260926 13:08:11.274876 24560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } }
I20260926 13:08:11.274952 24560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 13:08:11.274974 24560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 13:08:11.275002 24560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:11.275761 24560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } }
I20260926 13:08:11.275842 24560 leader_election.cc:302] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98334f719eaf4b2886dfb9fb456a2d55; no voters:
I20260926 13:08:11.275955 24560 leader_election.cc:288] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 13:08:11.276018 24563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:11.276204 24563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [term 1 LEADER]: Becoming Leader. State: Replica: 98334f719eaf4b2886dfb9fb456a2d55, State: Running, Role: LEADER
I20260926 13:08:11.276228 24560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:11.276291 24563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } }
I20260926 13:08:11.276629 24564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98334f719eaf4b2886dfb9fb456a2d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } } }
I20260926 13:08:11.276633 24565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98334f719eaf4b2886dfb9fb456a2d55. Latest consensus state: current_term: 1 leader_uuid: "98334f719eaf4b2886dfb9fb456a2d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98334f719eaf4b2886dfb9fb456a2d55" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 43885 } } }
I20260926 13:08:11.276732 24564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:11.276736 24565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:11.277202 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24492
I20260926 13:08:11.277302 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:11.277462 2392 external_mini_cluster.cc:1428] Setting key 5b1a23d304517a91c8c5ddf021eeecb6
I20260926 13:08:11.277647 24569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:11.278201 24569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:11.279726 24569 catalog_manager.cc:1347] Generated new cluster ID: 70120cf670c14474a3cb449ee4c5fd95
I20260926 13:08:11.279789 24569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:11.279927 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:11.278022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39631 (local address 127.2.86.62:43885)
0926 13:08:11.278195 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:11.278204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:11.278353 (+ 149us) server_negotiation.cc:407] Connection header received
0926 13:08:11.278510 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:11.278516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:11.278541 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:11.278675 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:11.279348 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:11.279352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:11.279352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:11.279356 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:11.279404 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:11.279419 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:11.279484 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:11.279596 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 13:08:11.279726 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260926 13:08:11.280738 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:11.283331 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:11.282232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42665 (local address 127.2.86.62:43885)
0926 13:08:11.282265 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:11.282267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:11.282372 (+ 105us) server_negotiation.cc:407] Connection header received
0926 13:08:11.282440 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:11.282441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:11.282446 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:11.282499 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:11.283094 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:11.283095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:11.283096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 13:08:11.283100 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:11.283112 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 13:08:11.283116 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:11.283157 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:11.283228 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 13:08:11.283242 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:08:11.288269 24569 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:11.288414 24569 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:11.301578 24569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98334f719eaf4b2886dfb9fb456a2d55: Generated new TSK 0
I20260926 13:08:11.301751 24569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:08:11.311609 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24492
2026-09-26T13:08:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1287 ms)
[----------] 18 tests from AuthTokenIssuingTest (23640 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:11 dist-test-slave-j4lq krb5kdc[24591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:11 dist-test-slave-j4lq krb5kdc[24591](info): set up 2 sockets
Sep 26 13:08:11 dist-test-slave-j4lq krb5kdc[24591](info): commencing operation
krb5kdc: starting...
W20260926 13:08:12.345130 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:12.459322 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42331
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:12.538079 24604 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:12.538303 24604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:12.538332 24604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:12.538375 24604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:12.539462 24604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:12.539526 24604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:12.539549 24604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:12.539579 24604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:12.539610 24604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:12.541070 24604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:42331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:12.541332 24604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:12.541560 24604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:12.546288 24609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:12.546288 24612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:12.546348 24604 server_base.cc:1034] running on GCE node
W20260926 13:08:12.546344 24610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:12.546768 24604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:12.547068 24604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:12.548211 24604 hybrid_clock.cc:648] HybridClock initialized: now 1790428092548193 us; error 36 us; skew 500 ppm
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:12.550572 24604 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:12.550951 24604 webserver.cc:460] Webserver started at http://127.2.86.62:37795/ using document root <none> and password file <none>
I20260926 13:08:12.551215 24604 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:12.551280 24604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:12.551424 24604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:12.552420 24604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "ddeb43bf65244fbab60e4c561a9e7a37"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "32e86f52d2cec7746f0a3be72ee8c529"
server_key_iv: "7ddde65e1d9d1f2ddaa48780aa111716"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.552766 24604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "ddeb43bf65244fbab60e4c561a9e7a37"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "32e86f52d2cec7746f0a3be72ee8c529"
server_key_iv: "7ddde65e1d9d1f2ddaa48780aa111716"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.554023 24604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:08:12.554725 24619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:12.554889 24604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:12.554980 24604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "ddeb43bf65244fbab60e4c561a9e7a37"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "32e86f52d2cec7746f0a3be72ee8c529"
server_key_iv: "7ddde65e1d9d1f2ddaa48780aa111716"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.555068 24604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:12.571004 24604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:12.572790 24604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:12.573014 24604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:12.576725 24604 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:42331
I20260926 13:08:12.576793 24671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:42331 every 8 connection(s)
I20260926 13:08:12.577154 24604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:12.578105 24672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:12.579674 24672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.583747 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24604
I20260926 13:08:12.583855 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:12.584088 2392 external_mini_cluster.cc:1428] Setting key 18c24578f8e4ed5e452011cd04c2ef03
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:12.589375 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.580307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:42331 (local address 127.0.0.1:44230)
0926 13:08:12.580459 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.580498 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.580997 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.581254 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.581256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.581503 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.581651 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.581655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.582592 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.582592 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.583250 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.583252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.583791 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.583792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.583886 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.584440 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.584449 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.585982 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.587800 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.587806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.587808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.588030 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.588282 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.588283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.588283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.588373 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.588724 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.588729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.588789 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.588932 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.589161 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260926 13:08:12.589376 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.580454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.2.86.62:42331)
0926 13:08:12.580588 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.580595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.580631 (+ 36us) server_negotiation.cc:407] Connection header received
0926 13:08:12.581131 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.581133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.581150 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.581229 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.581729 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.582514 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.583318 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.583690 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.583804 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.586098 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.586105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.586106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.586112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.587709 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.588131 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.588133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.588134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.588214 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.588438 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.588439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.588440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.588652 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.588746 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.588962 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.589100 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260926 13:08:12.590056 24672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.590411 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.590277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36150)
0926 13:08:12.590333 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.590385 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 13:08:12.590548 24620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
I20260926 13:08:12.590993 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.590873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36156)
0926 13:08:12.590932 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.590965 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 13:08:12.591138 24672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:43813: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.86.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
I20260926 13:08:12.608191 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:43813
--webserver_interface=127.2.86.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:08:12.640393 24672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } attempt: 1
I20260926 13:08:12.640954 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.640787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36166)
0926 13:08:12.640854 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.640920 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 13:08:12.641386 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.641292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36172)
0926 13:08:12.641335 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.641364 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 13:08:12.641510 24672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:43813: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
W20260926 13:08:12.688843 24679 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:12.689114 24679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:12.689132 24679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:12.689152 24679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:12.690296 24679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:12.690335 24679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:12.690349 24679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:12.690397 24679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:12.690415 24679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:12.691850 24679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.86.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:12.692087 24679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:12.692318 24679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:12.696967 24687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:12.696995 24679 server_base.cc:1034] running on GCE node
W20260926 13:08:12.696969 24684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:12.696993 24685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:12.697419 24679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:12.697693 24679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:12.698880 24679 hybrid_clock.cc:648] HybridClock initialized: now 1790428092698875 us; error 33 us; skew 500 ppm
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:12.701403 24679 init.cc:377] Logged in from keytab as kudu/127.2.86.61@KRBTEST.COM (short username kudu)
I20260926 13:08:12.701795 24679 webserver.cc:460] Webserver started at http://127.2.86.61:46283/ using document root <none> and password file <none>
I20260926 13:08:12.702031 24679 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:12.702100 24679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:12.702241 24679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:12.703234 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data/instance:
uuid: "b13b8e56c5394c13a5a8cbea8c2bac32"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "4c882f89c0a807c9a0bf3ae5622ec09a"
server_key_iv: "195b6c6dd260f921a50a6b5d28223588"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.703591 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal/instance:
uuid: "b13b8e56c5394c13a5a8cbea8c2bac32"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "4c882f89c0a807c9a0bf3ae5622ec09a"
server_key_iv: "195b6c6dd260f921a50a6b5d28223588"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.704859 24679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:08:12.705663 24694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:12.705873 24679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:12.705976 24679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "b13b8e56c5394c13a5a8cbea8c2bac32"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "4c882f89c0a807c9a0bf3ae5622ec09a"
server_key_iv: "195b6c6dd260f921a50a6b5d28223588"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.706058 24679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:12.709692 24672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } attempt: 2
I20260926 13:08:12.710069 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.709949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36184)
0926 13:08:12.710006 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.710040 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 13:08:12.710431 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.710352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36188)
0926 13:08:12.710391 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.710415 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 13:08:12.710525 24672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:43813: Network error: Client connection negotiation failed: client connection to 127.2.86.61:43813: connect: Connection refused (error 111)
I20260926 13:08:12.722508 24679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:12.724402 24679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:12.724507 24679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:12.728246 24679 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.61:43813
I20260926 13:08:12.728341 24746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.61:43813 every 8 connection(s)
I20260926 13:08:12.728638 24679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
I20260926 13:08:12.729637 24747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:12.731132 24747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.732808 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24679
I20260926 13:08:12.732924 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-1/wal/instance
I20260926 13:08:12.733124 2392 external_mini_cluster.cc:1428] Setting key 66a205a3ea822de38a9510cf4804eab0
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.60@KRBTEST.COM; defaulting to no policy
I20260926 13:08:12.740563 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.731790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.2.86.62:42331)
0926 13:08:12.731867 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.731869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.732029 (+ 160us) server_negotiation.cc:407] Connection header received
0926 13:08:12.732491 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.732492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.732504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.732546 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.733286 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.734035 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.735407 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.735608 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.735666 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.737819 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.737821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.737821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.737828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.739220 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.739625 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.739626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.739627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.739693 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.739938 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.739939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.739940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.740056 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.740132 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.740392 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.740473 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 13:08:12.740758 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.731747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:42331 (local address 127.0.0.1:44238)
0926 13:08:12.731918 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.731952 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.732388 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.732598 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.732601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.732839 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.733136 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.733141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.734129 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.734129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.735315 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.735318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.735710 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.735711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.735839 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.736358 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.736366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.737713 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.739320 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.739327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.739329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.739548 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.739760 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.739761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.739761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.739872 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.740123 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.740129 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.740267 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.740373 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.740522 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260926 13:08:12.741387 24747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.60@KRBTEST.COM" created.
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:12.748926 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.741740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.2.86.61:43813)
0926 13:08:12.741901 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.741904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.741919 (+ 15us) server_negotiation.cc:407] Connection header received
0926 13:08:12.741935 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.741936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.741947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.742026 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.742287 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.743096 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.743753 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.744069 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.744155 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.746197 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.746204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.746204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.746210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.747735 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.748136 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.748138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.748139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.748190 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.748393 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.748393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.748394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.748515 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.748572 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.748738 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.748804 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260926 13:08:12.749048 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.741677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36200)
0926 13:08:12.741712 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.741722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.741759 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.742026 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.742026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.742168 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.742213 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.742214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.743176 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.743177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.743661 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.743662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.744154 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.744155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.744224 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.744638 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.744642 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.746113 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.747819 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.747820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.747821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.748052 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.748255 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.748256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.748256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.748327 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.748577 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.748578 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.748628 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.748718 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.748893 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":54}
I20260926 13:08:12.749641 24747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.749997 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.749864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39682)
0926 13:08:12.749912 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.749967 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 13:08:12.750149 24698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
I20260926 13:08:12.750582 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.750466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39690)
0926 13:08:12.750520 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.750552 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 13:08:12.750752 24747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:39579: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
I20260926 13:08:12.759136 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:39579
--webserver_interface=127.2.86.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:46453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:08:12.799993 24747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } attempt: 1
I20260926 13:08:12.800545 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.800395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39700)
0926 13:08:12.800459 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.800508 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 13:08:12.800966 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.800844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39702)
0926 13:08:12.800913 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.800942 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 13:08:12.801079 24747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:39579: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
I20260926 13:08:12.801672 24672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } attempt: 3
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:12.808437 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.801977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.2.86.61:43813)
0926 13:08:12.802031 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.802033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.802045 (+ 12us) server_negotiation.cc:407] Connection header received
0926 13:08:12.802117 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.802118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.802128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.802178 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.802482 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.802979 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.803698 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.803912 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.803987 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.806054 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.806056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.806056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.806062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.807421 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.807792 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.807793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.807793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.807842 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.808029 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.808030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.808030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.808126 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.808172 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.808321 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.808381 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 13:08:12.808494 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.801927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36206)
0926 13:08:12.801973 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.801984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.802052 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.802182 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.802183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.802343 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.802408 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.802410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.803072 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.803073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.803602 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.803604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.803984 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.803985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.804062 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.804546 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.804551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.805971 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.807507 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.807508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.807508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.807725 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.807901 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.807902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.807902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.807966 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.808180 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.808180 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.808230 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.808299 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.808392 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:08:12.808916 24672 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.809273 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.809173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39710)
0926 13:08:12.809217 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.809252 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:08:12.809604 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.809515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39726)
0926 13:08:12.809556 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.809579 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 13:08:12.809693 24672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:39579: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
I20260926 13:08:12.840824 24672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } attempt: 1
I20260926 13:08:12.841279 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.841145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39734)
0926 13:08:12.841212 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.841250 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 13:08:12.841586 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.841493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39740)
0926 13:08:12.841534 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.841561 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 13:08:12.841687 24672 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:39579: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
W20260926 13:08:12.842087 24754 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:12.842306 24754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:12.842360 24754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:12.842406 24754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:12.843612 24754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:12.843674 24754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:12.843701 24754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:12.843732 24754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:12.843756 24754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:12.845234 24754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:46453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:42331,127.2.86.61:43813,127.2.86.60:39579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:39579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.86.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:12.845500 24754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:12.845713 24754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:12.850474 24759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:12.850536 24762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:12.850484 24760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:12.850689 24754 server_base.cc:1034] running on GCE node
I20260926 13:08:12.850912 24754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:12.851186 24754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:12.852370 24754 hybrid_clock.cc:648] HybridClock initialized: now 1790428092852354 us; error 30 us; skew 500 ppm
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:12.854871 24754 init.cc:377] Logged in from keytab as kudu/127.2.86.60@KRBTEST.COM (short username kudu)
I20260926 13:08:12.855304 24754 webserver.cc:460] Webserver started at http://127.2.86.60:41469/ using document root <none> and password file <none>
I20260926 13:08:12.855489 24754 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:12.855526 24754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:12.855623 24754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:12.856600 24754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data/instance:
uuid: "bf214af56bc2404bbc928391c1cc54d6"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "987d5f0d27d34483b737b2911e7e9433"
server_key_iv: "398481da4e7cae44ac74d5afb30c58ed"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.856966 24754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal/instance:
uuid: "bf214af56bc2404bbc928391c1cc54d6"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "987d5f0d27d34483b737b2911e7e9433"
server_key_iv: "398481da4e7cae44ac74d5afb30c58ed"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.858304 24754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:08:12.859064 24769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:12.859264 24754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:08:12.859357 24754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "bf214af56bc2404bbc928391c1cc54d6"
format_stamp: "Formatted at 2026-09-26 13:08:12 on dist-test-slave-j4lq"
server_key: "987d5f0d27d34483b737b2911e7e9433"
server_key_iv: "398481da4e7cae44ac74d5afb30c58ed"
server_key_version: "encryptionkey@0"
I20260926 13:08:12.859431 24754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:12.869251 24747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } attempt: 2
I20260926 13:08:12.869683 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.869543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39756)
0926 13:08:12.869605 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.869647 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 13:08:12.870059 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.869972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39768)
0926 13:08:12.870013 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.870036 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 13:08:12.870172 24747 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:39579: Network error: Client connection negotiation failed: client connection to 127.2.86.60:39579: connect: Connection refused (error 111)
I20260926 13:08:12.876178 24754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:12.878235 24754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:12.878408 24754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:12.882633 24754 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.60:39579
I20260926 13:08:12.882687 24821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.60:39579 every 8 connection(s)
I20260926 13:08:12.883036 24754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
I20260926 13:08:12.883744 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24754
I20260926 13:08:12.883859 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790427826505950-2392-0/minicluster-data/master-2/wal/instance
I20260926 13:08:12.884069 2392 external_mini_cluster.cc:1428] Setting key b25775270df96ea99d1d98bb3454be19
I20260926 13:08:12.884212 24822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:12.886690 24822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:12.895124 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.887332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44252 (local address 127.2.86.62:42331)
0926 13:08:12.887399 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.887401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.887590 (+ 189us) server_negotiation.cc:407] Connection header received
0926 13:08:12.887650 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.887651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.887661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.887702 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.888236 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.888830 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.889833 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.890008 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.890076 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.892285 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.892286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.892287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.892295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.893823 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.894217 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.894218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.894218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.894279 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.894539 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.894540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.894540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.894672 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.894742 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.894938 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.895018 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 13:08:12.895339 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.887276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:42331 (local address 127.0.0.1:44252)
0926 13:08:12.887452 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.887513 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.887589 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.887721 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.887723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.888070 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.888152 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.888154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.888939 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.888940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.889740 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.889743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.890109 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.890110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.890208 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.890654 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.890664 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.892194 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.893895 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.893902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.893904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.894148 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.894359 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.894360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.894361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.894466 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.894739 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.894744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.894800 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.894902 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.895074 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260926 13:08:12.895342 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.884634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.2.86.60:39579)
0926 13:08:12.884799 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.884808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.884846 (+ 38us) server_negotiation.cc:407] Connection header received
0926 13:08:12.885403 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.885410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.885460 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.885580 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.886352 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.887443 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.888468 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.889082 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.889158 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.891670 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.891679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.891679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.891691 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.893778 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.894224 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.894227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.894227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.894317 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.894539 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.894540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.894540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.894808 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.894936 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.895034 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.895141 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260926 13:08:12.895998 24822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:12.902591 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.896307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.2.86.61:43813)
0926 13:08:12.896350 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.896352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.896361 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:08:12.896386 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.896387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.896398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.896449 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.896716 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.897328 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.898023 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.898319 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.898366 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.900034 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.900035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.900036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.900042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.901503 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.901845 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.901846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.901846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.901904 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.902089 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.902090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.902090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.902195 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.902282 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.902440 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.902507 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:08:12.902591 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.896245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:43813 (local address 127.0.0.1:36214)
0926 13:08:12.896276 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.896288 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.896330 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.896452 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.896453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.896605 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.896631 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.896632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.897397 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.897398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.897950 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.897952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.898381 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.898382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.898437 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.898820 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.898824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.899953 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.901603 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.901604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.901604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.901772 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.901972 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.901972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.901973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.902031 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.902275 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.902276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.902315 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.902398 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.902502 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 13:08:12.902971 24822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:12.903445 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.896165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.2.86.62:42331)
0926 13:08:12.896198 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.896200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.896344 (+ 144us) server_negotiation.cc:407] Connection header received
0926 13:08:12.896422 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.896423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.896431 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.896483 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.897036 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.897520 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.898210 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.898400 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.898461 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.900890 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.900891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.900892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.900899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.902265 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.902627 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.902628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.902629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.902684 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.902937 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.902937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.902938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.903041 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.903223 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.903273 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.903348 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:12.909754 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.903201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39790)
0926 13:08:12.903244 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.903254 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.903287 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.903403 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.903404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.903662 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.903685 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.903687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.904463 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.904464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.905098 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.905100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.905491 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.905492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.905544 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.905859 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.905864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.907093 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.908615 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.908615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.908616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.908827 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.909114 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.909115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.909115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.909200 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.909430 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.909431 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.909467 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.909534 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.909632 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 13:08:12.909804 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.903260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.2.86.60:39579)
0926 13:08:12.903293 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.903295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.903303 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:08:12.903335 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.903336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.903353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.903401 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.903785 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.904390 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.905165 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.905422 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.905468 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.907181 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.907182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.907183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.907190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.908558 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.908922 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.908924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.908924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.909002 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.909266 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.909266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.909267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.909358 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.909437 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.909562 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.909680 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":18}
I20260926 13:08:12.911424 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.904073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.2.86.61:43813)
0926 13:08:12.904109 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.904110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.904118 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:08:12.904136 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.904137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.904144 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.904204 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.904673 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.905181 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.905953 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.906132 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.906191 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.908559 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.908560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.908561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.908568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.910322 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.910632 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.910634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.910634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.910697 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.910864 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.910865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.910866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.910985 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.911142 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.911184 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.911342 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 13:08:12.911474 24822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: Bootstrap starting.
I20260926 13:08:12.912477 24822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:12.912765 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:12.912801 24822 log.cc:824] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:12.913615 24822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: No bootstrap required, opened a new log
I20260926 13:08:12.913981 24646 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ddeb43bf65244fbab60e4c561a9e7a37"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266
I20260926 13:08:12.915711 24822 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.915856 24822 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf214af56bc2404bbc928391c1cc54d6, State: Initialized, Role: FOLLOWER
I20260926 13:08:12.915931 24822 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.916185 24828 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.916234 24822 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:12.916277 24828 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.916798 24672 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } attempt: 2
W20260926 13:08:12.918103 24839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:12.918195 24839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
I20260926 13:08:12.918817 24754 master_runner.cc:186] Error getting master registration for 127.2.86.62:42331: OK, instance_id { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" instance_seqno: 1790428092575705 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:08:12.921542 24754 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:08:12.924185 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.917089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39798)
0926 13:08:12.917135 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.917145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.917183 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.917397 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.917398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.917574 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.917622 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.917625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.918377 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.918378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.919145 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.919148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.919601 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.919602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.919674 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.920083 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.920088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.921434 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.923144 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.923145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.923145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.923334 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.923546 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.923546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.923547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.923607 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.923870 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.923871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.923912 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.924001 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.924093 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 13:08:12.924181 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.917243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.2.86.60:39579)
0926 13:08:12.917292 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.917294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.917301 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:08:12.917319 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.917319 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.917327 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.917381 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.917709 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.918289 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.919218 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.919515 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.919586 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.921520 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.921521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.921521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.921529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.923052 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.923411 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.923412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.923412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.923462 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.923677 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.923678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.923679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.923800 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.923877 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.924024 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.924093 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:08:12.925446 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: Bootstrap starting.
I20260926 13:08:12.925966 24672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:12.926203 24672 log.cc:824] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:12.926859 24672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: No bootstrap required, opened a new log
I20260926 13:08:12.927779 24672 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.927881 24672 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddeb43bf65244fbab60e4c561a9e7a37, State: Initialized, Role: FOLLOWER
I20260926 13:08:12.927974 24672 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.928259 24842 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.928323 24672 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:12.928336 24842 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.928412 24646 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 13:08:12.928532 24646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:12.929394 24646 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.929621 24646 leader_election.cc:288] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [CANDIDATE]: Term 1 election: Requested vote from peers b13b8e56c5394c13a5a8cbea8c2bac32 (127.2.86.61:43813), bf214af56bc2404bbc928391c1cc54d6 (127.2.86.60:39579)
I20260926 13:08:12.930084 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bf214af56bc2404bbc928391c1cc54d6"
I20260926 13:08:12.930207 24795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:12.931159 24795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddeb43bf65244fbab60e4c561a9e7a37 in term 1.
W20260926 13:08:12.931205 24853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:12.931983 24853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:12.931718 24623 leader_election.cc:302] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf214af56bc2404bbc928391c1cc54d6, ddeb43bf65244fbab60e4c561a9e7a37; no voters:
I20260926 13:08:12.932142 24842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:12.932281 24842 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [term 1 LEADER]: Becoming Leader. State: Replica: ddeb43bf65244fbab60e4c561a9e7a37, State: Running, Role: LEADER
I20260926 13:08:12.932369 24842 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.933506 24604 master_runner.cc:186] Error getting master registration for 127.2.86.61:43813: OK, instance_id { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" instance_seqno: 1790428092727250 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:08:12.935665 24604 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:08:12.936051 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddeb43bf65244fbab60e4c561a9e7a37. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.936261 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:12.936604 24864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:12.936916 24864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:12.941067 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.931203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.2.86.61:43813)
0926 13:08:12.931243 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.931246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.932293 (+ 1047us) server_negotiation.cc:407] Connection header received
0926 13:08:12.932396 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.932396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.932408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.932439 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.933297 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.934047 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.935043 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.935776 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.935824 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.937984 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.937986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.937986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.937994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.939751 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.940204 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.940205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.940206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.940265 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.940457 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.940458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.940458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.940624 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.940749 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.940900 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.940979 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 13:08:12.948282 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.930816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44272 (local address 127.2.86.62:42331)
0926 13:08:12.930856 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.930858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.931356 (+ 498us) server_negotiation.cc:407] Connection header received
0926 13:08:12.933043 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.933044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.933064 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.933095 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.940946 (+ 7851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.941612 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.942556 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.942837 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.942876 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.944455 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.944456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.944457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.944464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.946569 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.947162 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.947163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.947164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.947234 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.947541 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.947542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.947542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.947685 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.947768 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.948106 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.948192 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:08:12.948307 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.932017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.2.86.60:39579)
0926 13:08:12.932052 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.932053 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.941360 (+ 9307us) server_negotiation.cc:407] Connection header received
0926 13:08:12.941455 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.941456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.941466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.941495 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.942114 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.942799 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.943762 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.944057 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.944095 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.945523 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.945524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.945525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.945531 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.947127 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.947510 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.947511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.947511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.947561 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.947753 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.947753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.947754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.947867 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.947941 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.948133 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.948216 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:08:12.949733 24795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [term 1 FOLLOWER]: Refusing update from remote peer ddeb43bf65244fbab60e4c561a9e7a37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:08:12.950045 24862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:08:12.951398 24828 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddeb43bf65244fbab60e4c561a9e7a37. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.951486 24828 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.961318 24747 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } attempt: 3
I20260926 13:08:12.962072 24842 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.962168 24842 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:12.962323 24828 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.962399 24828 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.962677 24866 mvcc.cc:205] Tried to move back new op lower bound from 7333593468719759360 to 7333593468651675648. Current Snapshot: MvccSnapshot[applied={T|T < 7333593468719759360}]
I20260926 13:08:12.963460 24864 catalog_manager.cc:1347] Generated new cluster ID: 5a75a784e6764816ad87432e7ea6cc4d
I20260926 13:08:12.963505 24864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 13:08:12 dist-test-slave-j4lq krb5kdc[24591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
I20260926 13:08:12.970837 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.961571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:39579 (local address 127.0.0.1:39814)
0926 13:08:12.961655 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:12.961665 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:12.961701 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:12.962182 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:12.962184 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:12.962357 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:12.962403 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.962405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.963371 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.963372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.964182 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:12.964184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.964706 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.964707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:12.964763 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.965343 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:12.965348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:12.966873 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:12.968937 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.968939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.968939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.969171 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.970140 (+ 969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:12.970141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:12.970141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:12.970214 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:12.970518 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:12.970519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:12.970583 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 13:08:12.970644 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 13:08:12.970738 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 13:08:12.971187 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:12.961904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.2.86.60:39579)
0926 13:08:12.962050 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:12.962052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:12.962058 (+ 6us) server_negotiation.cc:407] Connection header received
0926 13:08:12.962075 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:12.962076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:12.962086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:12.962136 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:12.962500 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.963248 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.964288 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:12.964593 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:12.964652 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:12.966974 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:12.966975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:12.966976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:12.966982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:12.968815 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.969982 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.969983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.969983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.970056 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:12.970289 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:12.970290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:12.970290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:12.970425 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:12.970481 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:12.971029 (+ 548us) server_negotiation.cc:299] Negotiation successful
0926 13:08:12.971102 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260926 13:08:12.972579 24747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: Bootstrap starting.
I20260926 13:08:12.973299 24747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:12.973628 24747 log.cc:824] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:12.974530 24747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: No bootstrap required, opened a new log
I20260926 13:08:12.976492 24747 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.976645 24747 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13b8e56c5394c13a5a8cbea8c2bac32, State: Initialized, Role: FOLLOWER
I20260926 13:08:12.976727 24747 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } }
I20260926 13:08:12.977083 24747 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:12.977183 24720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:12.978040 24720 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [term 1 FOLLOWER]: Refusing update from remote peer ddeb43bf65244fbab60e4c561a9e7a37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:08:12.978070 24721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32"
I20260926 13:08:12.978422 24862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:08:12.978830 24721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddeb43bf65244fbab60e4c561a9e7a37 in term 1.
W20260926 13:08:12.981300 24886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:12.981354 24886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:12.981483 24871 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.981549 24871 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.981603 24871 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddeb43bf65244fbab60e4c561a9e7a37. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.981643 24871 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.981669 24871 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.981709 24871 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:12.985242 24879 mvcc.cc:205] Tried to move back new op lower bound from 7333593468719759360 to 7333593468651675648. Current Snapshot: MvccSnapshot[applied={T|T < 7333593468719759360}]
I20260926 13:08:12.990945 24862 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddeb43bf65244fbab60e4c561a9e7a37" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 42331 } } peers { permanent_uuid: "b13b8e56c5394c13a5a8cbea8c2bac32" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 43813 } } peers { permanent_uuid: "bf214af56bc2404bbc928391c1cc54d6" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 39579 } } }
I20260926 13:08:12.991040 24862 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37 [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:12.992697 24864 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:12.993429 24864 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:13.008699 24864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddeb43bf65244fbab60e4c561a9e7a37: Generated new TSK 0
I20260926 13:08:13.009086 24864 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:08:13.031934 24825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.030898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.2.86.60:39579)
0926 13:08:13.030941 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.030943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.031274 (+ 331us) server_negotiation.cc:407] Connection header received
0926 13:08:13.031380 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.031381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.031392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.031423 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.031870 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39824: BlockingRecv error: recv got EOF from 127.0.0.1:39824 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 13:08:13.033375 24675 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.032443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.2.86.62:42331)
0926 13:08:13.032482 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.032484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.032799 (+ 315us) server_negotiation.cc:407] Connection header received
0926 13:08:13.032940 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.032942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.032952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.032983 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.033322 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44276: BlockingRecv error: recv got EOF from 127.0.0.1:44276 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 13:08:13.034288 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.032678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.2.86.61:43813)
0926 13:08:13.032718 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.032720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.033766 (+ 1046us) server_negotiation.cc:407] Connection header received
0926 13:08:13.033865 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.033866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.033876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.033905 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.034227 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36244: BlockingRecv error: recv got EOF from 127.0.0.1:36244 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 13:08:13.036445 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.035715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.2.86.61:43813)
0926 13:08:13.035760 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.035762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.035948 (+ 186us) server_negotiation.cc:407] Connection header received
0926 13:08:13.036051 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.036051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.036067 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.036095 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.036407 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36258: BlockingRecv error: recv got EOF from 127.0.0.1:36258 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260926 13:08:13.039449 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.038702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.2.86.61:43813)
0926 13:08:13.038732 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.038734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.038935 (+ 201us) server_negotiation.cc:407] Connection header received
0926 13:08:13.039023 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.039024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.039033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.039061 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.039410 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36268: BlockingRecv error: recv got EOF from 127.0.0.1:36268 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 13:08:13.043313 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.042597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.2.86.61:43813)
0926 13:08:13.042630 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.042631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.042830 (+ 199us) server_negotiation.cc:407] Connection header received
0926 13:08:13.042924 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.042924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.042934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.042963 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.043273 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36278: BlockingRecv error: recv got EOF from 127.0.0.1:36278 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 13:08:13.049292 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.048467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36280 (local address 127.2.86.61:43813)
0926 13:08:13.048500 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.048501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.048704 (+ 203us) server_negotiation.cc:407] Connection header received
0926 13:08:13.048801 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.048802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.048811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.048839 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.049249 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36280: BlockingRecv error: recv got EOF from 127.0.0.1:36280 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 13:08:13.059620 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.058769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.2.86.61:43813)
0926 13:08:13.058834 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.058836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.059085 (+ 249us) server_negotiation.cc:407] Connection header received
0926 13:08:13.059195 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.059196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.059206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.059235 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.059569 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36296: BlockingRecv error: recv got EOF from 127.0.0.1:36296 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 13:08:13.078042 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.077120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.2.86.61:43813)
0926 13:08:13.077172 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.077174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.077442 (+ 268us) server_negotiation.cc:407] Connection header received
0926 13:08:13.077548 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.077549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.077561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.077591 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.077983 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36310: BlockingRecv error: recv got EOF from 127.0.0.1:36310 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 13:08:13.112269 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.111522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.2.86.61:43813)
0926 13:08:13.111568 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.111570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.111711 (+ 141us) server_negotiation.cc:407] Connection header received
0926 13:08:13.111784 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.111785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.111797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.111855 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.112195 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36318: BlockingRecv error: recv got EOF from 127.0.0.1:36318 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 13:08:13.178283 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.177498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.2.86.61:43813)
0926 13:08:13.177585 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.177587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.177621 (+ 34us) server_negotiation.cc:407] Connection header received
0926 13:08:13.177707 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.177708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.177774 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.177838 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.178219 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36320: BlockingRecv error: recv got EOF from 127.0.0.1:36320 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 13:08:13.308328 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.307521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.2.86.61:43813)
0926 13:08:13.307601 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.307604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.307715 (+ 111us) server_negotiation.cc:407] Connection header received
0926 13:08:13.307804 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.307805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.307823 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.307887 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.308263 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36330: BlockingRecv error: recv got EOF from 127.0.0.1:36330 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 13:08:13.566099 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:13.565457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.2.86.61:43813)
0926 13:08:13.565513 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:13.565515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:13.565550 (+ 35us) server_negotiation.cc:407] Connection header received
0926 13:08:13.565624 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:13.565625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:13.565635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:13.565709 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:13.566019 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36340: BlockingRecv error: recv got EOF from 127.0.0.1:36340 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 13:08:13.918648 24839 catalog_manager.cc:1259] Loaded cluster ID: 5a75a784e6764816ad87432e7ea6cc4d
I20260926 13:08:13.918716 24839 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: loading cluster ID for follower catalog manager: success
I20260926 13:08:13.919518 24839 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: acquiring CA information for follower catalog manager: success
I20260926 13:08:13.919819 24839 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf214af56bc2404bbc928391c1cc54d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:08:13.981745 24886 catalog_manager.cc:1259] Loaded cluster ID: 5a75a784e6764816ad87432e7ea6cc4d
I20260926 13:08:13.981841 24886 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: loading cluster ID for follower catalog manager: success
I20260926 13:08:13.982635 24886 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: acquiring CA information for follower catalog manager: success
I20260926 13:08:13.982945 24886 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b13b8e56c5394c13a5a8cbea8c2bac32: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:08:14.082015 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:14.079244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.2.86.61:43813)
0926 13:08:14.079358 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:14.079361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:14.079440 (+ 79us) server_negotiation.cc:407] Connection header received
0926 13:08:14.079549 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:14.079550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:14.079598 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:14.079683 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:14.079849 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.080347 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.081206 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.081406 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.081481 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:14.081665 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:14.081777 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:14.081867 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:14.081890 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 13:08:14.081911 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260926 13:08:14.085968 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:14.083765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.2.86.60:39579)
0926 13:08:14.083859 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:14.083860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:14.083870 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:08:14.083906 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:14.083907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:14.083940 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:14.084008 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:14.084123 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.084578 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.085305 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.085487 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.085536 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:14.085656 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:14.085747 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:14.085802 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:14.085838 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 13:08:14.085853 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":42,"threads_started":1}
I20260926 13:08:14.090396 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:14.087758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.2.86.61:43813)
0926 13:08:14.087805 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:14.087808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:14.087882 (+ 74us) server_negotiation.cc:407] Connection header received
0926 13:08:14.087979 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:14.087980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:14.087992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:14.088044 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:14.088158 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.088849 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.089626 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.089898 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.089962 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:14.090123 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:14.090157 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:14.090185 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:14.090280 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:08:14.090306 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:08:14.090605 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:14.087457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.2.86.62:42331)
0926 13:08:14.087604 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:14.087606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:14.087636 (+ 30us) server_negotiation.cc:407] Connection header received
0926 13:08:14.087764 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:14.087766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:14.087817 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:14.087903 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:14.088081 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.088588 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.089656 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.089974 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.090034 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:14.090240 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:14.090343 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:14.090404 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:14.090499 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 13:08:14.090528 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":57,"threads_started":1}
I20260926 13:08:14.090610 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:14.087589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.2.86.60:39579)
0926 13:08:14.087626 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:14.087628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:14.087793 (+ 165us) server_negotiation.cc:407] Connection header received
0926 13:08:14.087819 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:14.087820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:14.087830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:14.087876 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:14.088019 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.088755 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.089797 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:14.090055 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:14.090118 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:14.090249 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:14.090281 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:14.090326 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:14.090531 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 13:08:14.090554 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:08:14.092499 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24604
I20260926 13:08:14.097723 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24679
I20260926 13:08:14.102569 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24754
2026-09-26T13:08:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2792 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 13:08:14 dist-test-slave-j4lq krb5kdc[24972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 13:08:14 dist-test-slave-j4lq krb5kdc[24972](info): set up 2 sockets
Sep 26 13:08:14 dist-test-slave-j4lq krb5kdc[24972](info): commencing operation
krb5kdc: starting...
W20260926 13:08:15.145131 2392 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/test-user.keytab.
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T13:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T13:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.86.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
Entry for principal kudu/127.2.86.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab.
I20260926 13:08:15.259513 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45279
--webserver_interface=127.2.86.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 13:08:15.340852 24985 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:15.341135 24985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:15.341152 24985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:15.341219 24985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:15.342360 24985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:15.342397 24985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:15.342411 24985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:15.342458 24985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:15.342476 24985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:15.344023 24985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.62.keytab
--principal=kudu/127.2.86.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.62:45279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.86.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:15.344275 24985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:15.344507 24985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:15.349522 24990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:15.349607 24993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:15.349746 24985 server_base.cc:1034] running on GCE node
W20260926 13:08:15.349525 24991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:15.350073 24985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:15.350389 24985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:15.351547 24985 hybrid_clock.cc:648] HybridClock initialized: now 1790428095351529 us; error 30 us; skew 500 ppm
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:15.361069 24985 init.cc:377] Logged in from keytab as kudu/127.2.86.62@KRBTEST.COM (short username kudu)
I20260926 13:08:15.361546 24985 webserver.cc:460] Webserver started at http://127.2.86.62:36985/ using document root <none> and password file <none>
I20260926 13:08:15.361759 24985 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:15.361800 24985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:15.361889 24985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:15.362910 24985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data/instance:
uuid: "bbd514e954054d32a9e8904d262ae1bb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "36bcb70ad45295d59c3098db51550fd8"
server_key_iv: "5a70896449dfc3e66d0ebaf50c6fbb26"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.363205 24985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance:
uuid: "bbd514e954054d32a9e8904d262ae1bb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "36bcb70ad45295d59c3098db51550fd8"
server_key_iv: "5a70896449dfc3e66d0ebaf50c6fbb26"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.364567 24985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 13:08:15.365322 25000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:15.365468 24985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:15.365556 24985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal
uuid: "bbd514e954054d32a9e8904d262ae1bb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "36bcb70ad45295d59c3098db51550fd8"
server_key_iv: "5a70896449dfc3e66d0ebaf50c6fbb26"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.365610 24985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:15.376545 24985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:15.378345 24985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:15.378449 24985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:15.382315 24985 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.62:45279
I20260926 13:08:15.382377 25052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.62:45279 every 8 connection(s)
I20260926 13:08:15.382725 24985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/data/info.pb
I20260926 13:08:15.383689 25053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:15.384064 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 24985
I20260926 13:08:15.384150 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-0/wal/instance
I20260926 13:08:15.384369 2392 external_mini_cluster.cc:1428] Setting key 1c969d20fe78bfffb61ab2f17b7f25f2
I20260926 13:08:15.385414 25053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.86.61@KRBTEST.COM; defaulting to no policy
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.61@KRBTEST.COM" created.
I20260926 13:08:15.395622 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.386310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.2.86.62:45279)
0926 13:08:15.386447 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.386452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.386497 (+ 45us) server_negotiation.cc:407] Connection header received
0926 13:08:15.386923 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.386925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.386947 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.387051 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.387608 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.389055 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.389903 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.390411 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.390534 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.392537 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.392543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.392544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.392550 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.394162 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.394565 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.394568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.394569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.394644 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.394887 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.394888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.394888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.395087 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.395156 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.395371 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.395464 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260926 13:08:15.395748 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.386106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45279 (local address 127.0.0.1:48354)
0926 13:08:15.386268 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.386307 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.386769 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.387065 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.387067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.387332 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.387475 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.387479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.389149 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.389150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.389816 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.389818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.390521 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.390522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.390574 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.391102 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.391110 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.392460 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.394260 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.394266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.394268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.394470 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.394711 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.394712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.394712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.394814 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.395160 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.395167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.395243 (+ 76us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.395342 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.395567 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":58,"thread_start_us":94,"threads_started":1}
I20260926 13:08:15.396636 25053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:15.397018 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.396859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36594)
0926 13:08:15.396923 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.396985 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 13:08:15.397184 25002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
I20260926 13:08:15.397581 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.397475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36608)
0926 13:08:15.397520 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.397550 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 13:08:15.397691 25053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:34213: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
Entry for principal kudu/127.2.86.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab.
I20260926 13:08:15.409407 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:34213
--webserver_interface=127.2.86.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:08:15.446944 25053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } attempt: 1
I20260926 13:08:15.447522 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.447371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36624)
0926 13:08:15.447439 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.447489 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 13:08:15.447916 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.447824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36630)
0926 13:08:15.447870 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.447898 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:08:15.448037 25053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:34213: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
W20260926 13:08:15.489429 25060 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:15.489632 25060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:15.489650 25060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:15.489669 25060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:15.490804 25060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:15.490844 25060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:15.490859 25060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:15.490909 25060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:15.490936 25060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:15.492441 25060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.61.keytab
--principal=kudu/127.2.86.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.61:34213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.86.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:15.492681 25060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:15.492964 25060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:08:15.497637 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:15.497642 25068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:15.497895 25060 server_base.cc:1034] running on GCE node
W20260926 13:08:15.497648 25066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:15.498231 25060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:15.498536 25060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:15.499706 25060 hybrid_clock.cc:648] HybridClock initialized: now 1790428095499695 us; error 47 us; skew 500 ppm
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:15.502276 25060 init.cc:377] Logged in from keytab as kudu/127.2.86.61@KRBTEST.COM (short username kudu)
I20260926 13:08:15.502735 25060 webserver.cc:460] Webserver started at http://127.2.86.61:33287/ using document root <none> and password file <none>
I20260926 13:08:15.502977 25060 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:15.503046 25060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:15.503194 25060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:15.504252 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data/instance:
uuid: "4ced380d61de480d8471f01ba5922dd5"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "58b73150801a333bcfbf23b89550f0d5"
server_key_iv: "3197d2013f5a43118f918521f7d21e13"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.504609 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal/instance:
uuid: "4ced380d61de480d8471f01ba5922dd5"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "58b73150801a333bcfbf23b89550f0d5"
server_key_iv: "3197d2013f5a43118f918521f7d21e13"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.505997 25060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:08:15.506681 25075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:15.506855 25060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 13:08:15.506950 25060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal
uuid: "4ced380d61de480d8471f01ba5922dd5"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "58b73150801a333bcfbf23b89550f0d5"
server_key_iv: "3197d2013f5a43118f918521f7d21e13"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.507036 25060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:15.516206 25053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } attempt: 2
I20260926 13:08:15.516588 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.516454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36640)
0926 13:08:15.516509 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.516544 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 13:08:15.516955 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.516824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36642)
0926 13:08:15.516892 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.516917 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 13:08:15.517035 25053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.61:34213: Network error: Client connection negotiation failed: client connection to 127.2.86.61:34213: connect: Connection refused (error 111)
I20260926 13:08:15.523191 25060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:15.525470 25060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:15.525632 25060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:15.529529 25060 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.61:34213
I20260926 13:08:15.529564 25127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.61:34213 every 8 connection(s)
I20260926 13:08:15.529903 25060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/data/info.pb
I20260926 13:08:15.530818 25128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:15.532317 25128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:15.533788 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 25060
I20260926 13:08:15.533898 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-1/wal/instance
I20260926 13:08:15.534090 2392 external_mini_cluster.cc:1428] Setting key 729d1b7aaa301911e5950992bf7adaff
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.86.60@KRBTEST.COM; defaulting to no policy
I20260926 13:08:15.542601 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.532998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.2.86.62:45279)
0926 13:08:15.533063 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.533065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.533301 (+ 236us) server_negotiation.cc:407] Connection header received
0926 13:08:15.533782 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.533783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.533799 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.533872 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.534597 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.535360 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.536929 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.537139 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.537205 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.539747 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.539748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.539749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.539755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.541217 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.541687 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.541687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.541688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.541734 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.542006 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.542007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.542007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.542101 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.542150 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.542473 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.542539 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 13:08:15.542853 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.533001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45279 (local address 127.0.0.1:48370)
0926 13:08:15.533179 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.533213 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.533683 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.534006 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.534010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.534308 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.534495 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.534499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.535468 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.535469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.536810 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.536813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.537250 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.537251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.537397 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.537954 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.537965 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.539647 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.541301 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.541308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.541310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.541611 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.541801 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.541802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.541803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.541939 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.542162 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.542168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.542332 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.542458 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.542624 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260926 13:08:15.543391 25128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.86.60@KRBTEST.COM" created.
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:15.551302 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.543697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.2.86.61:34213)
0926 13:08:15.543837 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.543842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.543859 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:08:15.543886 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.543888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.543898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.543961 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.544156 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.545072 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.546102 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.546588 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.546666 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.548570 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.548576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.548576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.548582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.550177 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.550571 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.550573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.550573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.550622 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.550831 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.550831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.550831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.550947 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.551005 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.551171 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.551234 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260926 13:08:15.551433 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.543662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36652)
0926 13:08:15.543694 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.543704 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.543740 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.543964 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.543965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.544071 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.544115 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.544116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.545136 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.545137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.545997 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.546000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.546671 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.546672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.546724 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.547134 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.547138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.548489 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.550262 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.550264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.550264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.550482 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.550696 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.550696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.550697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.550762 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.551015 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.551016 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.551069 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.551151 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.551300 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":48}
I20260926 13:08:15.552083 25128 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:15.552433 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.552299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56186)
0926 13:08:15.552351 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.552404 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:08:15.552578 25078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
I20260926 13:08:15.553015 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.552928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56198)
0926 13:08:15.552962 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.552992 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260926 13:08:15.553170 25128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:44177: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
Entry for principal kudu/127.2.86.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab.
I20260926 13:08:15.559860 2392 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNbmtbn/build/release/bin/kudu
/tmp/dist-test-taskNbmtbn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:44177
--webserver_interface=127.2.86.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.86.20:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 13:08:15.602423 25128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } attempt: 1
I20260926 13:08:15.602949 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.602789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56206)
0926 13:08:15.602852 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.602914 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 13:08:15.603394 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.603295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56220)
0926 13:08:15.603342 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.603369 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 13:08:15.603495 25128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:44177: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
I20260926 13:08:15.608160 25053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } attempt: 3
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:15.615453 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.608436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36660)
0926 13:08:15.608488 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.608497 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.608560 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.608703 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.608704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.608858 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.608970 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.608972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.609815 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.609816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.610427 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.610430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.610889 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.610890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.610967 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.611457 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.611462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.612822 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.614426 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.614427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.614428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.614607 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.614808 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.614809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.614809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.614867 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.615125 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.615126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.615171 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.615259 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.615356 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 13:08:15.615448 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.608488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.2.86.61:34213)
0926 13:08:15.608543 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.608545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.608573 (+ 28us) server_negotiation.cc:407] Connection header received
0926 13:08:15.608620 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.608621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.608630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.608698 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.609074 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.609724 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.610516 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.610816 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.610891 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.612952 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.612953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.612954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.612960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.614339 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.614680 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.614681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.614682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.614736 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.614938 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.614939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.614940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.615058 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.615133 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.615282 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.615356 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 13:08:15.615895 25053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:15.616235 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.616123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56234)
0926 13:08:15.616174 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.616205 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 13:08:15.616509 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.616429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56236)
0926 13:08:15.616470 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.616494 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 13:08:15.616595 25053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:44177: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
W20260926 13:08:15.641129 25135 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 13:08:15.641358 25135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 13:08:15.641387 25135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 13:08:15.641427 25135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 13:08:15.642526 25135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 13:08:15.642589 25135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 13:08:15.642613 25135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 13:08:15.642634 25135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 13:08:15.642660 25135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 13:08:15.644119 25135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.86.20:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.86.62:45279,127.2.86.61:34213,127.2.86.60:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/krb5kdc/kudu_127.2.86.60.keytab
--principal=kudu/127.2.86.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.86.60:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.86.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 12:15:03 UTC on ccycloud.quasar-vyifkg.root.comops.site
build id 4674112
I20260926 13:08:15.644399 25135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:08:15.644621 25135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:08:15.647729 25053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } attempt: 1
I20260926 13:08:15.648451 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.648034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56242)
0926 13:08:15.648362 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.648409 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
I20260926 13:08:15.648962 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.648727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56244)
0926 13:08:15.648850 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.648899 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260926 13:08:15.649109 25053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:44177: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
W20260926 13:08:15.649188 25141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:15.649181 25140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:08:15.649428 25143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:08:15.649376 25135 server_base.cc:1034] running on GCE node
I20260926 13:08:15.649691 25135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 13:08:15.650002 25135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 13:08:15.651185 25135 hybrid_clock.cc:648] HybridClock initialized: now 1790428095651176 us; error 38 us; skew 500 ppm
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 13:08:15.653544 25135 init.cc:377] Logged in from keytab as kudu/127.2.86.60@KRBTEST.COM (short username kudu)
I20260926 13:08:15.653955 25135 webserver.cc:460] Webserver started at http://127.2.86.60:45507/ using document root <none> and password file <none>
I20260926 13:08:15.654152 25135 fs_manager.cc:359] Metadata directory not provided
I20260926 13:08:15.654194 25135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:08:15.654280 25135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:08:15.655257 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data/instance:
uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "4391e334727fb98f1bc746f0c9662d21"
server_key_iv: "3e0b6ae01a90ff86838e00d488f1b029"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.655544 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal/instance:
uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "4391e334727fb98f1bc746f0c9662d21"
server_key_iv: "3e0b6ae01a90ff86838e00d488f1b029"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.656744 25135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 13:08:15.657517 25150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:08:15.657685 25135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:08:15.657785 25135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data,/tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal
uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb"
format_stamp: "Formatted at 2026-09-26 13:08:15 on dist-test-slave-j4lq"
server_key: "4391e334727fb98f1bc746f0c9662d21"
server_key_iv: "3e0b6ae01a90ff86838e00d488f1b029"
server_key_version: "encryptionkey@0"
I20260926 13:08:15.657863 25135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:08:15.666664 25135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:08:15.668443 25135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:08:15.668612 25135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:08:15.671643 25128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } attempt: 2
I20260926 13:08:15.672086 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.671915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56246)
0926 13:08:15.671987 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.672035 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 13:08:15.672438 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.672344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56250)
0926 13:08:15.672389 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.672416 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 13:08:15.672578 25128 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.86.60:44177: Network error: Client connection negotiation failed: client connection to 127.2.86.60:44177: connect: Connection refused (error 111)
I20260926 13:08:15.672796 25135 rpc_server.cc:307] RPC server started. Bound to: 127.2.86.60:44177
I20260926 13:08:15.672868 25202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.86.60:44177 every 8 connection(s)
I20260926 13:08:15.673209 25135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/data/info.pb
I20260926 13:08:15.673975 2392 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNbmtbn/build/release/bin/kudu as pid 25135
I20260926 13:08:15.674094 2392 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNbmtbn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790427826505950-2392-0/minicluster-data/master-2/wal/instance
I20260926 13:08:15.674154 25203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:08:15.674328 2392 external_mini_cluster.cc:1428] Setting key 69bbc91e585593a531ed6cdae34c070b
I20260926 13:08:15.676433 25203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:15.685365 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.674813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.2.86.60:44177)
0926 13:08:15.674988 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.674998 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.675029 (+ 31us) server_negotiation.cc:407] Connection header received
0926 13:08:15.675557 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.675565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.675611 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.675722 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.676521 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.677594 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.678467 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.678966 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.679048 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.681904 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.681911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.681912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.681919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.683745 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684123 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.684125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.684126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.684199 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684395 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.684396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.684396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.684685 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.684817 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.684988 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.685087 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260926 13:08:15.685495 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.677095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.62:45279 (local address 127.0.0.1:48378)
0926 13:08:15.677289 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.677324 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.677414 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.677552 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.677554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.677878 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.677959 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.677961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.678692 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.678693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.679414 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.679416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.679839 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.679840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.679915 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.680314 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.680324 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.682431 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.684314 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.684321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.684571 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.684775 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.684775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.684841 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.685171 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.685177 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.685229 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.685293 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.685368 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260926 13:08:15.685547 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.677156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.2.86.62:45279)
0926 13:08:15.677222 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.677224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.677396 (+ 172us) server_negotiation.cc:407] Connection header received
0926 13:08:15.677467 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.677468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.677478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.677534 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.678048 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.678600 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.679500 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.679715 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.679800 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.682519 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.682520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.682521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.682528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.684214 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684640 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.684641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.684641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.684691 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.684943 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.684944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.684944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.685077 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.685142 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.685345 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.685430 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:08:15.686168 25203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.62@KRBTEST.COM
I20260926 13:08:15.692772 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.686392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.61:34213 (local address 127.0.0.1:36670)
0926 13:08:15.686431 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.686438 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.686467 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.686602 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.686603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.686733 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.686759 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.686760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.687368 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.687369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.687837 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.687838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.688220 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.688221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.688278 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.688654 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.688658 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.690012 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.691797 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.691798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.691798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.691951 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.692157 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.692158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.692158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.692206 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.692463 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.692464 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.692506 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.692595 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.692683 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 13:08:15.692780 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.686445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.2.86.61:34213)
0926 13:08:15.686495 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.686497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.686506 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:08:15.686529 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.686529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.686538 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.686591 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.686836 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.687286 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.687928 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.688136 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.688210 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.690099 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.690100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.690101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.690108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.691718 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.692026 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.692027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.692028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.692088 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.692266 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.692267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.692268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.692408 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.692468 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.692620 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.692696 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 13:08:15.693112 25203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:08:15.693593 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.686321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.2.86.62:45279)
0926 13:08:15.686354 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.686355 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.686363 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:08:15.686426 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.686426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.686436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.686490 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.686966 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.687401 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.688063 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.688258 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.688330 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.690868 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.690869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.690869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.690875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.692303 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.692625 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.692627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.692627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.692679 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.692946 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.692947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.692948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.693090 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.693219 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.693426 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.693496 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.60@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.86.61@KRBTEST.COM
I20260926 13:08:15.700055 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.693362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56278)
0926 13:08:15.693402 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.693411 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.693443 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.693815 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.693816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.693958 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.693986 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.693988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.694780 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.694781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.695237 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.695238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.695606 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.695607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.695655 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.696035 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.696040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.697443 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.699063 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.699064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.699065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.699223 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.699428 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.699429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.699429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.699480 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.699736 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.699737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.699772 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.699855 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.699947 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 13:08:15.700109 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.693393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.2.86.60:44177)
0926 13:08:15.693709 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.693711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.693718 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:08:15.693736 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.693737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.693746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.693805 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.694243 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.694682 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.695316 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.695519 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.695606 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.697518 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.697519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.697519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.697525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.698967 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.699306 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.699307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.699307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.699359 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.699549 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.699550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.699550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.699650 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.699732 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.699887 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.699996 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":296}
I20260926 13:08:15.701035 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.694206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.2.86.61:34213)
0926 13:08:15.694244 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.694246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.694255 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:08:15.694296 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.694297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.694306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.694359 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.694798 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.695233 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.695859 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.696074 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.696155 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.698484 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.698485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.698485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.698492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.699861 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.700282 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.700284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.700284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.700343 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.700514 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.700515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.700515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.700622 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.700802 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.700836 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.700940 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 13:08:15.701630 25203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: Bootstrap starting.
I20260926 13:08:15.702477 2392 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 13:08:15.702685 25203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:15.702962 25203 log.cc:824] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:15.703714 25203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: No bootstrap required, opened a new log
I20260926 13:08:15.703522 25027 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bbd514e954054d32a9e8904d262ae1bb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48380
I20260926 13:08:15.705250 25203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.705366 25203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3c5d0136eb4b1ead54e2bdc20feadb, State: Initialized, Role: FOLLOWER
I20260926 13:08:15.705452 25203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.705785 25203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:15.705775 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.705919 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: This master's current role is: FOLLOWER
W20260926 13:08:15.707588 25220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:15.707638 25220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:15.708017 25135 master_runner.cc:186] Error getting master registration for 127.2.86.62:45279: OK, instance_id { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" instance_seqno: 1790428095381121 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:08:15.710870 25135 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:08:15.724283 25053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } attempt: 2
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.62@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
I20260926 13:08:15.731714 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.724626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56282)
0926 13:08:15.724689 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.724704 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.724767 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.724947 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.724948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.725117 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.725159 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.725167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.725814 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.725815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.726305 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.726306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.726686 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.726687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.726742 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.727234 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.727240 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.728749 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.730568 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.730569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.730570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.730798 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.731005 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.731006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.731006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.731080 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.731392 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.731393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.731440 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.731516 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.731611 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 13:08:15.731729 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.724710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.2.86.60:44177)
0926 13:08:15.724788 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.724790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.724800 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:08:15.724826 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.724827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.724838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.724922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.725254 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.725724 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.726389 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.726594 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.726686 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.728827 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.728828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.728828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.728835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.730482 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.730876 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.730877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.730878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.730935 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.731146 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.731146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.731147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.731323 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.731388 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.731547 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.731623 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 13:08:15.733403 25053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: Bootstrap starting.
I20260926 13:08:15.734148 25053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:15.734448 25053 log.cc:824] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:15.735356 25053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: No bootstrap required, opened a new log
I20260926 13:08:15.736717 25053 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.736887 25053 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd514e954054d32a9e8904d262ae1bb, State: Initialized, Role: FOLLOWER
I20260926 13:08:15.736994 25053 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.737349 25053 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:15.737324 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.737418 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.737506 25027 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 13:08:15.737571 25027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:15.738354 25027 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.738860 25027 leader_election.cc:288] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [CANDIDATE]: Term 1 election: Requested vote from peers 4ced380d61de480d8471f01ba5922dd5 (127.2.86.61:34213), 5e3c5d0136eb4b1ead54e2bdc20feadb (127.2.86.60:44177)
W20260926 13:08:15.739539 25234 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:08:15.739742 25234 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:15.739797 24985 master_runner.cc:186] Error getting master registration for 127.2.86.61:34213: OK, instance_id { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" instance_seqno: 1790428095528313 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 13:08:15.742605 24985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 13:08:15.743173 25177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd514e954054d32a9e8904d262ae1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb"
I20260926 13:08:15.743299 25177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:15.744204 25177 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd514e954054d32a9e8904d262ae1bb in term 1.
I20260926 13:08:15.744445 25003 leader_election.cc:302] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e3c5d0136eb4b1ead54e2bdc20feadb, bbd514e954054d32a9e8904d262ae1bb; no voters:
I20260926 13:08:15.744562 25223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:08:15.744701 25223 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [term 1 LEADER]: Becoming Leader. State: Replica: bbd514e954054d32a9e8904d262ae1bb, State: Running, Role: LEADER
I20260926 13:08:15.744779 25223 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.745223 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd514e954054d32a9e8904d262ae1bb. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.745323 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:15.753545 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.742695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.2.86.61:34213)
0926 13:08:15.742753 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.742755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.742765 (+ 10us) server_negotiation.cc:407] Connection header received
0926 13:08:15.742791 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.742792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.742803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.742860 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.744228 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.745018 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.746854 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.747612 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.747660 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.750167 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.750168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.750168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.750176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.751833 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.752392 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.752393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.752394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.752451 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.752751 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.752764 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.752765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.752915 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.753011 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.753333 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.753414 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 13:08:15.753913 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.742838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.2.86.60:44177)
0926 13:08:15.742934 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.742936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.742943 (+ 7us) server_negotiation.cc:407] Connection header received
0926 13:08:15.742961 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.742962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.742971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.743020 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.746181 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.746856 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.747858 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.748488 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.748531 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.750797 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.750799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.750799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.750806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.752660 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.753068 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.753069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.753069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.753123 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.753321 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.753322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.753322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.753438 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.753511 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.753748 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.753821 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 13:08:15.755507 25245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:08:15.755738 25245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:08:15.757412 25177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [term 1 FOLLOWER]: Refusing update from remote peer bbd514e954054d32a9e8904d262ae1bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:08:15.758245 25243 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:08:15.759526 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd514e954054d32a9e8904d262ae1bb. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.759630 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.760337 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.742743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.2.86.62:45279)
0926 13:08:15.742780 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.742782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.753006 (+ 10224us) server_negotiation.cc:407] Connection header received
0926 13:08:15.753109 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.753110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.753119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.753150 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.753798 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.754491 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.755400 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.755708 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.755779 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.757246 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.757247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.757247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.757255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.758878 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.759264 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.759265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.759265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.759322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.759513 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.759513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.759514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.759638 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.759753 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.760138 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.760245 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 13:08:15.761874 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.761956 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:15.762126 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.762202 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.762295 25246 mvcc.cc:205] Tried to move back new op lower bound from 7333593480219488256 to 7333593480171433984. Current Snapshot: MvccSnapshot[applied={T|T < 7333593480219488256}]
I20260926 13:08:15.762384 25247 mvcc.cc:205] Tried to move back new op lower bound from 7333593480219488256 to 7333593480171433984. Current Snapshot: MvccSnapshot[applied={T|T < 7333593480219488256}]
I20260926 13:08:15.763733 25128 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } attempt: 3
Sep 26 13:08:15 dist-test-slave-j4lq krb5kdc[24972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790428095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.86.61@KRBTEST.COM for kudu/127.2.86.60@KRBTEST.COM
I20260926 13:08:15.771945 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.763957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.86.60:44177 (local address 127.0.0.1:56304)
0926 13:08:15.763999 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 13:08:15.764011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 13:08:15.764058 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 13:08:15.764195 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 13:08:15.764196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 13:08:15.764356 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0926 13:08:15.764398 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.764399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.765307 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.765308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.766129 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 13:08:15.766131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.766826 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.766827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 13:08:15.766892 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.767441 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 13:08:15.767446 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 13:08:15.769050 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 13:08:15.770819 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.770820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.770821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.771044 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.771266 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 13:08:15.771267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 13:08:15.771268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 13:08:15.771330 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 13:08:15.771623 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 13:08:15.771623 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 13:08:15.771671 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 13:08:15.771740 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 13:08:15.771842 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 13:08:15.772285 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:15.764005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.2.86.60:44177)
0926 13:08:15.764046 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.764048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.764065 (+ 17us) server_negotiation.cc:407] Connection header received
0926 13:08:15.764120 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.764120 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.764130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.764186 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.764473 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.765227 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.766227 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:15.766524 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:15.766591 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:15.769153 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 13:08:15.769154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 13:08:15.769154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 13:08:15.769161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 13:08:15.770734 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.771124 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.771125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.771126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.771186 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 13:08:15.771404 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 13:08:15.771405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 13:08:15.771406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 13:08:15.771548 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 13:08:15.771619 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:15.772119 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 13:08:15.772197 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 13:08:15.774086 25128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: Bootstrap starting.
I20260926 13:08:15.774809 25128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: Neither blocks nor log segments found. Creating new log.
I20260926 13:08:15.775128 25128 log.cc:824] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: Log is configured to *not* fsync() on all Append() calls
I20260926 13:08:15.776014 25128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: No bootstrap required, opened a new log
I20260926 13:08:15.778194 25128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.778370 25128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ced380d61de480d8471f01ba5922dd5, State: Initialized, Role: FOLLOWER
I20260926 13:08:15.778470 25128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } }
I20260926 13:08:15.778789 25128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:08:15.778887 25102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd514e954054d32a9e8904d262ae1bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4ced380d61de480d8471f01ba5922dd5"
I20260926 13:08:15.779003 25102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:08:15.779810 25102 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd514e954054d32a9e8904d262ae1bb in term 1.
I20260926 13:08:15.780237 25101 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [term 1 FOLLOWER]: Refusing update from remote peer bbd514e954054d32a9e8904d262ae1bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:08:15.780525 25243 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:08:15.781350 25245 catalog_manager.cc:1347] Generated new cluster ID: a68e041f6d6d49ae9048432b8675c624
I20260926 13:08:15.781402 25245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:08:15.782351 25265 mvcc.cc:205] Tried to move back new op lower bound from 7333593480219488256 to 7333593480171433984. Current Snapshot: MvccSnapshot[applied={T|T < 7333593480219488256}]
I20260926 13:08:15.782641 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.782747 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.782830 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd514e954054d32a9e8904d262ae1bb. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.782917 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.783715 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.783792 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:08:15.789662 25263 catalog_manager.cc:1259] Loaded cluster ID: a68e041f6d6d49ae9048432b8675c624
I20260926 13:08:15.789780 25263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: loading cluster ID for follower catalog manager: success
W20260926 13:08:15.789904 25263 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 13:08:15.789968 25263 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:08:15.814330 25243 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bbd514e954054d32a9e8904d262ae1bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbd514e954054d32a9e8904d262ae1bb" member_type: VOTER last_known_addr { host: "127.2.86.62" port: 45279 } } peers { permanent_uuid: "4ced380d61de480d8471f01ba5922dd5" member_type: VOTER last_known_addr { host: "127.2.86.61" port: 34213 } } peers { permanent_uuid: "5e3c5d0136eb4b1ead54e2bdc20feadb" member_type: VOTER last_known_addr { host: "127.2.86.60" port: 44177 } } }
I20260926 13:08:15.814463 25243 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb [sys.catalog]: This master's current role is: LEADER
I20260926 13:08:15.816999 25245 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:08:15.817777 25245 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:08:15.829897 25245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbd514e954054d32a9e8904d262ae1bb: Generated new TSK 0
I20260926 13:08:15.830129 25245 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 13:08:15.851297 25206 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.849678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.2.86.60:44177)
0926 13:08:15.849719 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.849722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.850643 (+ 921us) server_negotiation.cc:407] Connection header received
0926 13:08:15.850762 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.850763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.850773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.850804 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.851231 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56316: BlockingRecv error: recv got EOF from 127.0.0.1:56316 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 13:08:15.852167 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.850058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.2.86.61:34213)
0926 13:08:15.850094 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.850096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.851639 (+ 1543us) server_negotiation.cc:407] Connection header received
0926 13:08:15.851753 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.851754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.851765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.851794 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.852106 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36704: BlockingRecv error: recv got EOF from 127.0.0.1:36704 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 13:08:15.853008 25057 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.850383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.2.86.62:45279)
0926 13:08:15.850420 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.850422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.852467 (+ 2045us) server_negotiation.cc:407] Connection header received
0926 13:08:15.852563 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.852564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.852574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.852602 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.852946 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48394: BlockingRecv error: recv got EOF from 127.0.0.1:48394 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 13:08:15.858121 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.857239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.2.86.61:34213)
0926 13:08:15.857274 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.857276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.857509 (+ 233us) server_negotiation.cc:407] Connection header received
0926 13:08:15.857612 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.857613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.857631 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.857672 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.857994 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36712: BlockingRecv error: recv got EOF from 127.0.0.1:36712 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 13:08:15.861300 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.860474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.2.86.61:34213)
0926 13:08:15.860507 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.860509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.860780 (+ 271us) server_negotiation.cc:407] Connection header received
0926 13:08:15.860903 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.860904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.860915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.860945 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.861257 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36716: BlockingRecv error: recv got EOF from 127.0.0.1:36716 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 13:08:15.865288 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.864490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.2.86.61:34213)
0926 13:08:15.864530 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.864532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.864764 (+ 232us) server_negotiation.cc:407] Connection header received
0926 13:08:15.864857 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.864858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.864905 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.864935 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.865248 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36726: BlockingRecv error: recv got EOF from 127.0.0.1:36726 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260926 13:08:15.871173 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.870414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.2.86.61:34213)
0926 13:08:15.870446 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.870448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.870682 (+ 234us) server_negotiation.cc:407] Connection header received
0926 13:08:15.870782 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.870782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.870793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.870822 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.871130 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36740: BlockingRecv error: recv got EOF from 127.0.0.1:36740 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 13:08:15.886116 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.885168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.2.86.61:34213)
0926 13:08:15.885215 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.885217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.885520 (+ 303us) server_negotiation.cc:407] Connection header received
0926 13:08:15.885625 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.885626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.885638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.885688 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.886057 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36752: BlockingRecv error: recv got EOF from 127.0.0.1:36752 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 13:08:15.904337 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.903543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.2.86.61:34213)
0926 13:08:15.903586 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.903588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.903762 (+ 174us) server_negotiation.cc:407] Connection header received
0926 13:08:15.903802 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.903803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.903816 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.903888 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.904279 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36768: BlockingRecv error: recv got EOF from 127.0.0.1:36768 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 13:08:15.938280 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:15.937534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.2.86.61:34213)
0926 13:08:15.937594 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:15.937597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:15.937698 (+ 101us) server_negotiation.cc:407] Connection header received
0926 13:08:15.937784 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:15.937786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:15.937798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:15.937875 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:15.938221 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36770: BlockingRecv error: recv got EOF from 127.0.0.1:36770 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 13:08:16.004205 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:16.003508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.2.86.61:34213)
0926 13:08:16.003577 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.003579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.003648 (+ 69us) server_negotiation.cc:407] Connection header received
0926 13:08:16.003691 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.003692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.003701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.003768 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:16.004120 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36772: BlockingRecv error: recv got EOF from 127.0.0.1:36772 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 13:08:16.134143 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:16.133480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.2.86.61:34213)
0926 13:08:16.133535 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.133537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.133616 (+ 79us) server_negotiation.cc:407] Connection header received
0926 13:08:16.133675 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.133676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.133686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.133728 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:16.134078 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36788: BlockingRecv error: recv got EOF from 127.0.0.1:36788 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260926 13:08:16.391913 25132 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 13:08:16.391286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.2.86.61:34213)
0926 13:08:16.391350 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.391352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.391408 (+ 56us) server_negotiation.cc:407] Connection header received
0926 13:08:16.391456 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.391457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.391466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.391527 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 13:08:16.391857 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36796: BlockingRecv error: recv got EOF from 127.0.0.1:36796 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 13:08:16.708050 25220 catalog_manager.cc:1259] Loaded cluster ID: a68e041f6d6d49ae9048432b8675c624
I20260926 13:08:16.708118 25220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: loading cluster ID for follower catalog manager: success
I20260926 13:08:16.708941 25220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: acquiring CA information for follower catalog manager: success
I20260926 13:08:16.709241 25220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e3c5d0136eb4b1ead54e2bdc20feadb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:08:16.791018 25263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: acquiring CA information for follower catalog manager: success
I20260926 13:08:16.791399 25263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4ced380d61de480d8471f01ba5922dd5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:08:16.907694 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:16.905224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.2.86.61:34213)
0926 13:08:16.905351 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.905353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.905361 (+ 8us) server_negotiation.cc:407] Connection header received
0926 13:08:16.905388 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.905388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.905437 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.905525 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:16.905655 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.906146 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.906978 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.907171 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.907226 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:16.907362 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:16.907479 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:16.907538 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:16.907568 (+ 30us) server_negotiation.cc:299] Negotiation successful
0926 13:08:16.907586 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260926 13:08:16.911650 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:16.909545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.2.86.60:44177)
0926 13:08:16.909640 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.909642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.909651 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:08:16.909685 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.909686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.909716 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.909783 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:16.909862 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.910357 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.911052 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.911234 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.911285 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:16.911398 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:16.911486 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:16.911541 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:16.911574 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 13:08:16.911592 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":48,"threads_started":1}
I20260926 13:08:16.916105 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:16.913213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.2.86.61:34213)
0926 13:08:16.913338 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.913340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.913531 (+ 191us) server_negotiation.cc:407] Connection header received
0926 13:08:16.913611 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.913612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.913622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.913686 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:16.913915 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.914607 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915337 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.915617 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915681 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:16.915838 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:16.915870 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:16.915899 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:16.916016 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 13:08:16.916039 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 13:08:16.916208 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:16.913299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.2.86.60:44177)
0926 13:08:16.913334 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.913336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.913563 (+ 227us) server_negotiation.cc:407] Connection header received
0926 13:08:16.913684 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.913685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.913694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.913740 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:16.913881 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.914362 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915402 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.915692 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915750 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:16.915909 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:16.915940 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:16.916000 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:16.916098 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 13:08:16.916118 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 13:08:16.916487 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 13:08:16.913243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.2.86.62:45279)
0926 13:08:16.913388 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 13:08:16.913390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 13:08:16.913399 (+ 9us) server_negotiation.cc:407] Connection header received
0926 13:08:16.913492 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 13:08:16.913493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 13:08:16.913540 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 13:08:16.913614 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 13:08:16.913755 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.914520 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915492 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 13:08:16.915749 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 13:08:16.915816 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 13:08:16.916001 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 13:08:16.916099 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 13:08:16.916161 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 13:08:16.916375 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 13:08:16.916400 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":51,"threads_started":1}
I20260926 13:08:16.918030 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 24985
I20260926 13:08:16.923475 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 25060
I20260926 13:08:16.928182 2392 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNbmtbn/build/release/bin/kudu with pid 25135
2026-09-26T13:08:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2825 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5617 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (270428 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST